Home› Brickset.com
Shopping at LEGO or Amazon?
Please use our links: LEGO.com • Amazon
As an Amazon Associate we earn from qualifying purchases.

API v3 discussion

2

Comments

  • benbacardibenbacardi EnglandMember Posts: 712
    @Huw - something I noticed when playing with the API - #21020 is classified as "Architecture" with a subtheme of "Landmark series" whereas all the others in that subtheme are "Landmark Series" - note the case difference of "series" vs "Series"! Here's the JSON:
          "setID": 23406,
      "number": "21020",
      "numberVariant": 1,
      "name": "Trevi Fountain",
      "year": 2014,
      "theme": "Architecture",
      "themeGroup": "Model making",
      "subtheme": "Landmark series",
    vs 
          "setID": 6679,
      "number": "21000",
      "numberVariant": 1,
      "name": "Sears Tower",
      "year": 2008,
      "theme": "Architecture",
      "themeGroup": "Model making",
      "subtheme": "Landmark Series",
    It doesn't make any material difference, but unless I ignore case when grouping sets (which is easily done!) then the Trevi Fountain gets a group all of its own.
    This doesn't manifest on Brickset itself, so the logic there must be case insensitive. Not sure if it's something you care about fixing?
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    I've edited it.

    SQL server is case-insensitive and we display themes/subthemes etc. in upper case so is not noticeable.
    benbacardi
  • nicmarenicmare BerlinMember Posts: 13
    Played around with 

    getKeyUsageStats

    works good! thank you @Huw ! very helpful!
    Huw
  • nicmarenicmare BerlinMember Posts: 13
    Also increased caching methods on my side. 
  • hantothantot UKMember Posts: 284
    might be a push to move to v3 but has getSets on v2 stopped accepting multiple set numbers? can get single set data back fine but not multiple ones
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    edited May 2020
    ^ Yes it has. I've been simplifying the stored procedure to speed it up.

  • RexfelisRexfelis Snohomish, WAMember Posts: 32
    Any chance of bringing back "getReviews" in the v3 API?
  • RexfelisRexfelis Snohomish, WAMember Posts: 32
    Also, a more obscure one, any chance of adding back parameters to "getThemes" and "getSubthemes" to filter by user(Hash), "owned" and "wanted"? (like the v2 getThemesForUser and getSubthemesForUser)
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    edited May 2020
    They were not used, and I think the v2 versions have been disabled.

    I'm reluctant to add every feature to the API: its purpose is not to facilitate the production of something that negates the need to visit the website, but to supplement it and encourage visiting/registering.
    RikTheVeggie
  • RexfelisRexfelis Snohomish, WAMember Posts: 32
    A bit disappointed, but I understand your reasoning.

    If I ever get my app close to release, maybe we can figure out some ways to drive some income to the website while still providing a native app experience (and make it more worthwhile to invest in the API).
  • RexfelisRexfelis Snohomish, WAMember Posts: 32
    Uhhhh. I think I broke something?



    I was updating my app to use the v3 API and was at the point of updating/validating my use of setCollection when I noticed that I was getting duplicate results in my getSets response. At first I thought is was just a bug on my app, but decided to check the website as well and this is what showed up...

    Or is it maybe some weird local caching thing for me?
  • RexfelisRexfelis Snohomish, WAMember Posts: 32
    For the record, it seemed to be working fine when I was still (mistakenly, I thought) sending the parameters in the POST body (as JSON)...it didn't seem to go wonky until I updated to pass the parameters in the query string instead of the post body.
  • RexfelisRexfelis Snohomish, WAMember Posts: 32
    Okay, odd. It seems to have fixed itself now that I was able to make a successful setCollection update using the API test page.

    Apparently maybe I'm just doing it a bit wrong. Anyhow, seems to be mostly working now. So now the only problem I'm having is that I can't send an empty string for "notes". 
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    The notes issue is a known bug: I suggest sending a single space until I resolve it!
  • seb303seb303 UKMember Posts: 2
    For part of my application, the user can save groups of sets.  I would like to be able to retrieve all the set data with a single call to getSets, passing multiple setIDs.  From the API docs this doesn't seem possible.  Is there a way to pass a list of setIDs as criteria to getSets?  e.g. as a JSON array rather than a single integer?



  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    Not currently but it's a good idea...
  • seb303seb303 UKMember Posts: 2
    edited June 2020
    So currently I'll have to call getSets multiple times.  I will implement this for now, but will change over to using a single call once it's possible.
    Thank you.
  • LebosteinLebostein GermanyMember Posts: 53
    edited August 2020
    On the set site I see the UVP for France:

    But if I use the API I don't get this information:

        "LEGOCom": {
          "US": {
            "retailPrice": 99.99,
            "dateFirstAvailable": "2020-07-03T00:00:00Z"
          },
          "UK": {
            "retailPrice": 89.99,
            "dateFirstAvailable": "2020-07-01T00:00:00Z",
            "dateLastAvailable": "2020-07-03T00:00:00Z"
          },
          "CA": {
            "retailPrice": 149.99,
            "dateFirstAvailable": "2020-07-01T00:00:00Z"
          },
          "DE": {
            "retailPrice": 97.47,
            "dateFirstAvailable": "2020-07-02T00:00:00Z",
            "dateLastAvailable": "2020-07-04T00:00:00Z"
          }
        }

    Why?

  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    FR is not currently exposed via the API. Neither is NL, FI, SE, DK, NO etc. etc. 

    Although the price will be different the dates will be the same.
  • benbacardibenbacardi EnglandMember Posts: 712
    Out of curiosity, which field are searched when you pass a "query" to getSets? I presume it's a case-insensitive "contains" query against number and name, not anything more complex?
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    Number, name, theme and subtheme.

    benbacardi
  • LebosteinLebostein GermanyMember Posts: 53
    Huw said:
    FR is not currently exposed via the API. Neither is NL, FI, SE, DK, NO etc. etc. 

    Although the price will be different the dates will be the same.

    Is there a reason for that? I thought the web page and the API are based on the same database?

  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    I guess I don't want to give all my hard-earned data away ;)
  • louislllouisll USMember Posts: 1
    Hi! Thank you for providing all the LEGO data! It's super helpful. Currently, I am using the API for a student web application project. I found there is a limit of API call for getSet, could someone help me to increase the API call limit? Thank you! @Huw
  • RikTheVeggieRikTheVeggie CanadaMember Posts: 356
    @Huw assume you're aware that the API is down atm and https://brickset.com/api/v3.asmx results in an error page?  I'm guessing this is related to recent stability issues and it's been taken offline temporarily to reduce load on the site.
    It's been problematic all day, but didn't want to bug you until the site stabilised.
    canon03
  • canon03canon03 USAMember Posts: 361
    ^Thanks for reporting this.  I thought it was just me.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    @RikTheVeggie, yes that is exactly why it's down. The server has been stable overnight so I will re-enable it shortly.
    RikTheVeggiecanon03
  • LebosteinLebostein GermanyMember Posts: 53
    If I try to use the API with
    I see the error page:

  • YellowcastleYellowcastle Northern VirginiaAdministrator, Moderator Posts: 5,234
    @Lebostein - please see above
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    It's still unavailable while I gather more data about the server's performance without it.

    Apologies for the inconvenience.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    I'll be re-enabling the API in the morning, when I can keep an eye on the effect it's happening.

    I may have have to implement rate limiting if it causes issues.
  • RikTheVeggieRikTheVeggie CanadaMember Posts: 356
    Thanks @Huw!

    I'm already caching most data on my own server and within the app to minimise calls to getSets, but if you're seeing a lot of hits please LMK and I'll see what I can do.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    It's now back online.
    nicmareRikTheVeggiecanon03
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    @RikTheVeggie, your key is still the most used, with around 30,000 calls a day, which is around 20 a minute.

    However, that's nothing compared to the traffic I am seeing from what I assume is a rogue bot, which is making 150,000 page requests a day, or 100 a minute. I've now blocked it in CloudFlare and the server seems to be much happier now!
    MaffyDAstrobricks
  • RikTheVeggieRikTheVeggie CanadaMember Posts: 356
    Thanks for bringing the API back online @Huw.

    I'll take another walk through the code and see if I can reduce that number a bit.
  • nicmarenicmare BerlinMember Posts: 13
    i still have problems to make database requests. i can do this by hand when inserting the url in the browser. works fine. but by code it does not work. last time this happened i was because of cloudflare blocking it. and indeed this time its cloudflare again:

    Error 1020

    Ray ID: 675c9e9f6aa505bb • 2021-07-28 08:08:26 UTC

    Access denied

    What happened?

    This website is using a security service to protect itself from online attacks.

    Cloudflare Ray ID: 675c9e9f6aa505bb • Your IP: 87.171.71.242 • Performance & security by Cloudflare

    hantotHavok
  • HavokHavok FRMember Posts: 3
    Hi,
    it seems it not work anymore for me too since API is online again. 
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    edited July 2021
    The main keys are recording usage so I think it's working OK.

    @nicmare, That ray ID tells me that CloudFlare has invoked its 'manage definite bots' firewall rule for some reason.
  • RikTheVeggieRikTheVeggie CanadaMember Posts: 356
    For me, it looks to be working online and from mobile device, but the debugger is getting HTTP 403 (resource is forbidden) which generally means the CAPTCHA is required.
  • HavokHavok FRMember Posts: 3
    For me it’s not working since api is back online. But i’m in holydys and can see what happens exactly. IP server adress is 51.68.11.211 i don’t know if that can help. I’m restricted to 100 calls by days so i don’t think i overload the server :)
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    Please try https://brickset.com/api/v3.1.asmx and let me know how you get on. It's identical, just a different endpoint.
    RikTheVeggie
  • RikTheVeggieRikTheVeggie CanadaMember Posts: 356
    The debugger still gives a 403 for https://brickset.com/api/v3.1.asmx/checkKey and https://brickset.com/api/v3.1.asmx/login :( 
    Error stream returned is "error code: 1020"
  • canon03canon03 USAMember Posts: 361
    I am getting the same "error code: 1020" as @RikTheVeggie using api/v3.asmx and api/v3.1.asmx.  I can however successfully test my API key through https://brickset.com/api/v3.asmx?op=getSets.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    That's a nuisance. It looks like it's CloudFlare rules that are the issue, then. Send me some Ray IDs and I'll investigate further.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 7,059
    I've just found a setting that might help: I've told CloudFlare not to block 'Definitely automated' bot traffic. 

    LMK...
    canon03nicmareRikTheVeggie
  • hantothantot UKMember Posts: 284
    Huw said:
    I've just found a setting that might help: I've told CloudFlare not to block 'Definitely automated' bot traffic. 

    LMK...
    thats worked for me
  • canon03canon03 USAMember Posts: 361
    It is also working for me again.  Thanks @Huw!
  • RikTheVeggieRikTheVeggie CanadaMember Posts: 356
    Same here.  Getting HTTP 200 for both v3 and v3.1, all unit tests are now working.
    Thanks @Huw!
  • HavokHavok FRMember Posts: 3
    That’s working for me too ! Thanks a lot !
  • nicmarenicmare BerlinMember Posts: 13
    works here too now. thank you @Huw !
Sign In or Register to comment.

Shopping at LEGO.com or Amazon?

Please use our links: LEGO.com • Amazon

Recent discussions • Categories • Privacy Policy • Brickset.com

Howdy, Stranger!

It looks like you're new here. Sign in or register to get started.

Sign In Register

Categories

Brickset.com is a participant in the Amazon Services LLC Associates Program, the Amazon.com.ca, Inc. Associates Program and the Amazon EU Associates Programme, which are affiliate advertising programs designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.

As an Amazon Associate we earn from qualifying purchases.