Shopping at LEGO or Amazon?
Please use our links: LEGO.comAmazon
As an Amazon Associate we earn from qualifying purchases.

API v3 discussion

LebosteinLebostein Member Posts: 54
edited April 2020 in Brickset.com
With API v3 there is no method "getSet" (Retrive full information about one set) to get these special fields...
«13

Comments

  • RexfelisRexfelis Member Posts: 32
    Hey guys, sorry again for being so demanding. Just trying to make my mobile app as good as I can manage. :)

    Anyhow, a few more things I'd love to be able to do through the API:
    1. Add totalDistinctMinifigsOwned to getCollectionTotals
    2. Add support to getSets for getting sets by tag
    3. A getYears/getAllYears service that returns a list of all years, with numbers of sets/number of themes for each year
    4. More info in getCollectionTotals (additional items from https://brickset.com/mycollection summary - rankings, total piece count, total retail value)

    Thanks again,
        Leif
  • HuwHuw Administrator Posts: 7,119
    edited March 2020
    Pass a setID to getGets and it'll behave the same as the old getGet.
  • LebosteinLebostein Member Posts: 54
    edited March 2020
    Works. Thanks!
    By the way: What is the reason for that? So the users of my tool always trigger 2 API requests internally: The first request to get the setID from the entered set number and the second request to get all information from the setID. This doubles the number of requests and the download time....

  • HuwHuw Administrator Posts: 7,119
    edited March 2020
    It would have been the same in v2: getSet returned one record with extended data, which required a setID, obtained from a list of sets from getSets.

    However, I'll change the code so that specifying a set number will also return the extended data.

    Give me a day or two to upload it.
  • LebosteinLebostein Member Posts: 54
    edited March 2020
    That would be great! That would speed up the query a lot.
    What about a new json parameter for getSets to request the full information? {'extendedData':true}
  • RexfelisRexfelis Member Posts: 32
    Resurrecting an old thread, to see if any of this would be possible in the new v3 API?
  • HuwHuw Administrator Posts: 7,119
    1. I've not transferred that method across to v3 yet, it sees very little use, just 75 calls out of 1.5 million in the last month, so I wasn't going to bother.

    2. Yes I can do that

    3. Good idea, I'll have the method accept a blank theme and return a complete list

    4. See 1.
  • HuwHuw Administrator Posts: 7,119
    2 and 3 have been done on my dev machine and I'll upload in the next few days with the next update.
    paulmison
  • LebosteinLebostein Member Posts: 54
    edited April 2020
    It seems it works now. I have seen a new parameter "extendedData" and I get the extended data, but it doesn't seem to matter when it's on or off...

  • HuwHuw Administrator Posts: 7,119
    If the result set  contains just one record the extended fields will be returned anyway. It's only useful if you want all the data in a large data set.
  • LebosteinLebostein Member Posts: 54
    Nice! Thanks!
  • RikTheVeggieRikTheVeggie Member Posts: 356
    Just wanted to say thanks @Huw for the new API and having a fast turnaround on any tweaks.

    The extendedData field will prove useful, I can use that when pulling in recent sets which means I no longer need an extra call per set to get the description :)
  • RexfelisRexfelis Member Posts: 32
    Awesome, thank you!
  • RexfelisRexfelis Member Posts: 32
    And yeah, I can understand the reluctance on #1 and #4. I'm just kind of a geek about tracking the number of things in my collection...

    (and it fleshes out the UI in my app a bit, so there's more to show for the users "Profile")
  • HuwHuw Administrator Posts: 7,119
    Good, and you're welcome...
  • LebosteinLebostein Member Posts: 54
    How I increase the API calls per day from 100 to an other level? I have no idea, how many users use my software. If I suppose that 100 users per day updates their complete set library (on average perhaps 200 sets), then we'll be up to 20000 ...
    nicmare
  • nicmarenicmare Member Posts: 13
    Lebostein said:
    How I increase the API calls per day from 100 to an other level? I have no idea, how many users use my software. If I suppose that 100 users per day updates their complete set library (on average perhaps 200 sets), then we'll be up to 20000 ...
    would like to know this as well
  • LebosteinLebostein Member Posts: 54
    Can I see how many API calls were triggered with my old v2 api key per day?
  • nicmarenicmare Member Posts: 13
    Why is there a remove date when requesting 42111?
    {"status":"success","matches":1,"sets":[
      {
        "setID": 30040,
        "number": "42111",
        "numberVariant": 1,
        "name": "Dom's Dodge Charger",
        "year": 2020,
        "theme": "Technic",
        "themeGroup": "Technical",
        "pieces": 1077,
        "image": true,
        "thumbnailURL": "https://images.brickset.com/sets/small/42111-1.jpg",
        "imageURL": "https://images.brickset.com/sets/images/42111-1.jpg",
        "bricksetURL": "https://brickset.com/sets/42111-1",
        "released": true,
        "owned": false,
        "wanted": false,
        "qtyOwned": 0,
        "userRating": 0,
        "userNotes": "",
        "ownedByTotal": 19,
        "wantedByTotal": 468,
        "UKRetailPrice": 89.99,
        "USRetailPrice": 99.99,
        "CARetailPrice": 139.99,
        "EURetailPrice": 99.99,
        "dateAddedToLEGOdotComUSA": "2020-03-31T00:00:00Z",
        "dateRemovedFromLEGOdotComUSA": "2020-04-01T00:00:00Z",
        "rating": 0.0,
        "packagingType": "Box",
        "availability": "Retail - limited",
        "instructionsCount": 0,
        "additionalImageCount": 21,
        "ageMin": 10,
        "category": "Normal",
        "tags": "Car, Sprung, Hot Rod, 20 Wide Car, Functional Steering, Large Scale Vehicle, ",
        "EAN": "5702016617498",
        "lastUpdated": "2020-03-31T14:48:55.567Z"
      }
    ]}
  • HuwHuw Administrator Posts: 7,119
    It no longer appears in the affiliate product feed, it looks as if you can't pre-order it in the USA any more, but you could for a day or so.

    I should really put UK/DE dates in the feed, too.
  • HuwHuw Administrator Posts: 7,119
    The limit is only on the GetSets method and it's only there so I can keep a tab on the apps out there that are using the API because at the moment I have no clue who's doing what with v2.

    The only requirement to have the cap lifted is to provide information so I can maintain this list: https://brickset.com/tools/mobileapps. So, email me, or send me a PM.

    myBrickset, the heaviest user, typically calls the method fewer than 10,000 times a day so I've set its limit to 20,000. I'd do the same for yours.

    @Lebostein, yes I can tell you. PM me your api key.
  • HuwHuw Administrator Posts: 7,119
    nicmare, I'll add DE and UK dates with the next update.
    nicmare
  • HuwHuw Administrator Posts: 7,119
    edited April 2020
    I added the additional fields and realised that the sets class was becoming a mess and some structure was needed.

    There's a window of opportunity to sort it before anyone uses it for production purposes, so I thought I'd sort it now before it's too late.

    It will look like this once I've pushed the code to the server, possibly tomorrow morning.

    I welcome your comments before I do so.
    {"status":"success","matches":1,"sets":[
      {
        "setID": 28404,
        "number": "60223",
        "numberVariant": 1,
        "name": "Harvester Transport",
        "year": 2019,
        "theme": "City",
        "themeGroup": "Modern day",
        "subtheme": "Great Vehicles",
        "category": "Normal",
        "released": true,
        "pieces": 358,
        "minifigs": 3,
        "image": {
          "thumbnailURL": "https://images.brickset.com/sets/small/60223-1.jpg",
          "imageURL": "https://images.brickset.com/sets/images/60223-1.jpg"
        },
        "bricksetURL": "https://brickset.com/sets/60223-1",
        "collection": {
          "owned": false,
          "wanted": false,
          "qtyOwned": 0,
          "rating": 0,
          "notes": ""
        },
        "collections": {
          "ownedBy": 3869,
          "wantedBy": 1159
        },
        "LEGOCom": {
          "US": {
            "RetailPrice": 29.99,
            "dateFirstAvailable": "2019-01-01T00:00:00Z",
            "dateLastAvailable": "2019-12-28T00:00:00Z"
          },
          "UK": {
            "RetailPrice": 24.99,
            "dateFirstAvailable": "2018-12-27T00:00:00Z",
            "dateLastAvailable": "2019-12-02T00:00:00Z"
          },
          "CA": {
            "RetailPrice": 39.99,
            "dateFirstAvailable": "2018-12-02T00:00:00Z",
            "dateLastAvailable": "2019-12-28T00:00:00Z"
          },
          "DE": {
            "RetailPrice": 29.99,
            "dateFirstAvailable": "2018-12-27T00:00:00Z",
            "dateLastAvailable": "2019-12-02T00:00:00Z"
          }
        },
        "rating": 5.0,
        "packagingType": "Box",
        "availability": "Retail",
        "instructionsCount": 6,
        "additionalImageCount": 6,
        "ageRange": {
          "min": 5
        },
        "dimensions": {
          "height": 18.9,
          "width": 35.1,
          "depth": 8.8,
          "weight": 0.649
        },
        "barcode": {
          "EAN": "673419303668"
        },
        "extendedData": {
          "tags": "Truck, Trailer, Farm, Articulated Lorry, Transport Vehicle, Combine Harvester, ",
          "description": "<p>Help the LEGO&reg; City farmer manage his crops with the 60223 Harvester Transport! This heavy-duty toy transport truck features an opening minifigure cab and a hitch, plus a flatbed truck transport trailer with a holder for the harvester tool and 2 lowering ramps. This fantastic farm toy also features a white, yellow and green toy combine harvester with an opening minifigure cab, removable harvester tool and articulated unloading auger at the back, plus 2 LEGO minifigures and a buildable scarecrow figure.</p>\n\n<ul>\n\t<li>Includes 2 LEGO&reg; City minifigures: a farmer and a truck driver, plus a buildable scarecrow figure.</li>\n\t<li>Features a large Harvester Transport toy truck with opening minifigure cab, dual exhaust pipes and a hitch, transport trailer with holder for the harvester tool, connector to hold the harvester in place and 2 lowering ramps.</li>\n\t<li>Also features a combine harvester with an opening minifigure cab, removable harvesting tool, articulated unloading auger at the back, and a connector to secure it to the transport trailer.</li>\n\t<li>Accessory elements include a mug and a pitchfork.</li>\n\t<li>Unhook the transport trailer from the toy truck and lower the ramps to load the harvester.</li>\n\t<li>Remove the harvesting tool from the front of the toy combine harvester and attach it to the trailer.</li>\n\t<li>Secure the unloading auger and lock the combine harvester into place on the trailer.</li>\n\t<li>Raise the ramps, hook the transport trailer to the toy truck and haul the harvester away.</li>\n\t<li>Harvester Transport with trailer measures over 3&rdquo; (8cm) high, 12&rdquo; (31cm) long and 2&rdquo; (7cm) wide.</li>\n\t<li>Combine Harvester measures over 3&rdquo; (8cm) high, 6&rdquo; (17cm) long and 3&rdquo; (10cm) wide.</li>\n\t<li>Scarecrow stands over 1&rdquo; (5cm) tall.</li>\n</ul>\n"
        },
        "lastUpdated": "2019-08-01T23:51:26.13Z"
      }
    ]}
  • benbacardibenbacardi Member Posts: 712
    JSON is definitely nicer to work with than XML!

    Is there a reason to have "tags" be a string with comma-separated items, rather than an actual list/array of individual tags?
  • HuwHuw Administrator Posts: 7,119
    Good point -- No good reason, it's only like that because that's how it was in v2.

    I can certainly change it if there are no objections.
    benbacardi
  • HuwHuw Administrator Posts: 7,119
    edited April 2020
    Sorted...

    {"status":"success","matches":1,"sets":[
      {
        "extendedData": {
          "tags": [
            "Truck",
            "Trailer",
            "Farm",
            "Articulated Lorry",
            "Transport Vehicle",
            "Combine Harvester"
          ],
          "description": "<p>Help the LEGO&reg; City farmer manage his crops with the 60223 Harvester Transport! This heavy-duty toy transport truck features an opening minifigure cab and a hitch, plus a flatbed truck transport trailer with a holder for the harvester tool and 2 lowering ramps. This fantastic farm toy also features a white, yellow and green toy combine harvester with an opening minifigure cab, removable harvester tool and articulated unloading auger at the back, plus 2 LEGO minifigures and a buildable scarecrow figure.</p>\n\n<ul>\n\t<li>Includes 2 LEGO&reg; City minifigures: a farmer and a truck driver, plus a buildable scarecrow figure.</li>\n\t<li>Features a large Harvester Transport toy truck with opening minifigure cab, dual exhaust pipes and a hitch, transport trailer with holder for the harvester tool, connector to hold the harvester in place and 2 lowering ramps.</li>\n\t<li>Also features a combine harvester with an opening minifigure cab, removable harvesting tool, articulated unloading auger at the back, and a connector to secure it to the transport trailer.</li>\n\t<li>Accessory elements include a mug and a pitchfork.</li>\n\t<li>Unhook the transport trailer from the toy truck and lower the ramps to load the harvester.</li>\n\t<li>Remove the harvesting tool from the front of the toy combine harvester and attach it to the trailer.</li>\n\t<li>Secure the unloading auger and lock the combine harvester into place on the trailer.</li>\n\t<li>Raise the ramps, hook the transport trailer to the toy truck and haul the harvester away.</li>\n\t<li>Harvester Transport with trailer measures over 3&rdquo; (8cm) high, 12&rdquo; (31cm) long and 2&rdquo; (7cm) wide.</li>\n\t<li>Combine Harvester measures over 3&rdquo; (8cm) high, 6&rdquo; (17cm) long and 3&rdquo; (10cm) wide.</li>\n\t<li>Scarecrow stands over 1&rdquo; (5cm) tall.</li>\n</ul>\n"
        },
        "lastUpdated": "2019-08-01T23:51:26.13Z"
      }
    ]}
    benbacardi
  • LebosteinLebostein Member Posts: 54
    edited April 2020
    Could you add a "version" field or a new api method, that we can check wich version the data sructure is? So it is possible for me to convert old data structures to new and so we can better handle further changes in data structure.... or are only changes in data structure planed, if the api version number changed?
  • masterX244masterX244 Member Posts: 536
    Thats the logoc behind semantic versioning. Once a major version is locked in breaking changes require a new version number. Currently V3 seems to be in prerelease state where it is not locked in yet
    benbacardi
  • HuwHuw Administrator Posts: 7,119
    I do not anticipate changing the structure any more. Maybe adding to it, but that won't be a problem for anyone. 
  • HuwHuw Administrator Posts: 7,119
    I'll publish the changes tomorrow morning.
  • LebosteinLebostein Member Posts: 54
    edited April 2020

    <image>true/false</image> and <imageFilename>6510-1</imageFilename> is missing in the new structure...or were these fields removed?




  • HuwHuw Administrator Posts: 7,119
    imageFilename is redundant and you can take the presence of a imageURL to determine if there is one.
  • HuwHuw Administrator Posts: 7,119
    Changes are now live, and documentation updated.
  • LebosteinLebostein Member Posts: 54
    Thanks!

    For my internal converter, can you confirm, that the following XML fields are removed:
    • image (Boolean)
    • imageFilename (String) Returned only if an image is available (image=true)
    • largeThumbnailURL (String) Returned only if an image is available (image=true). Max dimensions 240x240
    • ACMDataCount (Integer) Number of ACM records user has modified for this set
    for all other fields I have found an assignment...







  • LebosteinLebostein Member Posts: 54
    With the new API about every second or third call is extremely slow (> 10 seconds). Whether I try it with the browser or with my programming language. I didn't have such delays with the old API. What can be the reason for this?


  • nwr122nwr122 Member Posts: 150
    @Huw Is is it possible to have current value returned within a set response?
  • HuwHuw Administrator Posts: 7,119
    Sorry for the delay in replying...

    @Lebostein, correct, those fields are redundant. I am aware of the speed issue and I'm working on it.

    @nwr122, I haven't included that in the API because it's not my data to give away. I get it from BrickLink's API.



  • nwr122nwr122 Member Posts: 150
    Huw said:

    @nwr122, I haven't included that in the API because it's not my data to give away. I get it from BrickLink's API.



    Thanks @Huw, thought that might be the case
  • benbacardibenbacardi Member Posts: 712
    I had a brief play with the API the other day (I've never used it before.) I was trying to get a list of sets without specifying a userHash (which the docs say is optional.)

    I found that if I didn't specify the userHash parameter at all, I got a 500 Server Error, and I needed to specify it as an empty string to retrieve sets with no user data. 

    Is a 500 intentional? I would have expected a 400 for a Bad Request instead, the 500 implies something went wrong on the server end. 
  • LebosteinLebostein Member Posts: 54
    I have measured the speed last days. One API call takes about 1 second. After about 25 calls there was a slow call with 20 seconds (!!) and so on. But since today or yesterday the speed is OK. Slow calls I had no more....


  • masterX244masterX244 Member Posts: 536
    I had a brief play with the API the other day (I've never used it before.) I was trying to get a list of sets without specifying a userHash (which the docs say is optional.)

    I found that if I didn't specify the userHash parameter at all, I got a 500 Server Error, and I needed to specify it as an empty string to retrieve sets with no user data. 

    Is a 500 intentional? I would have expected a 400 for a Bad Request instead, the 500 implies something went wrong on the server end. 

    For me a 500 looks like a case missed by the developer since those are usually caused by a uncaught exception somewhere. (had my fair share of 500s due to stupid errors as a developer already)
  • benbacardibenbacardi Member Posts: 712
    ^ Yes, that's usually what a 500 means. Just bringing it to Huw's attention unless he's already aware!
  • HuwHuw Administrator Posts: 7,119
    Lebostein said:
    I have measured the speed last days. One API call takes about 1 second. After about 25 calls there was a slow call with 20 seconds (!!) and so on. But since today or yesterday the speed is OK. Slow calls I had no more....


    Good, because I uploaded new code this morning to address it.

  • HuwHuw Administrator Posts: 7,119
    edited April 2020

    I found that if I didn't specify the userHash parameter at all, I got a 500 Server Error, and I needed to specify it as an empty string to retrieve sets with no user data. 

    Is a 500 intentional? I would have expected a 400 for a Bad Request instead, the 500 implies something went wrong on the server end. 

    It seems to be an anomaly with the MS web services framework in that all parameters need to be present, if only left blank.

    It reports the following and I'm not sure if I can change the response but I'll look into it.

    System.InvalidOperationException: Missing parameter: userHash.
       at System.Web.Services.Protocols.ValueCollectionParameterReader.Read(NameValueCollection collection)
       at System.Web.Services.Protocols.UrlParameterReader.Read(HttpRequest request)
       at System.Web.Services.Protocols.HttpServerProtocol.ReadParameters()
       at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()
    benbacardi
  • LebosteinLebostein Member Posts: 54
    edited April 2020
    Did you speed it up any more? At the moment I have on average only 0.7 seconds per call. Nice!


  • HuwHuw Administrator Posts: 7,119
    I've been optimising all the frequently used stored procedures so overall it may well have caused another performance boost.
  • masterX244masterX244 Member Posts: 536
    Huw said:

    I found that if I didn't specify the userHash parameter at all, I got a 500 Server Error, and I needed to specify it as an empty string to retrieve sets with no user data. 

    Is a 500 intentional? I would have expected a 400 for a Bad Request instead, the 500 implies something went wrong on the server end. 

    It seems to be an anomaly with the MS web services framework in that all parameters need to be present, if only left blank.

    It reports the following and I'm not sure if I can change the response but I'll look into it.

    System.InvalidOperationException: Missing parameter: userHash.
       at System.Web.Services.Protocols.ValueCollectionParameterReader.Read(NameValueCollection collection)
       at System.Web.Services.Protocols.UrlParameterReader.Read(HttpRequest request)
       at System.Web.Services.Protocols.HttpServerProtocol.ReadParameters()
       at System.Web.Services.Protocols.WebServiceHandler.CoreProcessRequest()
    Bugreport time to Microsoft then...

  • LebosteinLebostein Member Posts: 54
    edited April 2020
    Is the old API 2 broken? Should work until October I heard... I get a bunch of wrong sets if I use getSets with a specific number like "60249-1"...


  • HuwHuw Administrator Posts: 7,119
    There was a bug but it's sorted now.

    BTW, I'm adding a new method getKeyUsageStats that'll provide a means of seeing how many calls have been made to getSets using your key each day.

    It may well expand to cover more data if and when I start recording it!
    canon03benbacardinicmare
  • benbacardibenbacardi Member Posts: 712
    Exactly what I was wishing was available the other day! Great :)
Sign In or Register to comment.

Shopping at LEGO.com or Amazon?

Please use our links: LEGO.com Amazon

Recent discussions Categories Privacy Policy Brickset.com

Howdy, Stranger!

It looks like you're new here. Sign in or register to get started.

Brickset.com is a participant in the Amazon Services LLC Associates Program, the Amazon.com.ca, Inc. Associates Program and the Amazon EU Associates Programme, which are affiliate advertising programs designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.

As an Amazon Associate we earn from qualifying purchases.