Your help needed to fill the gaps in the parts database

HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,440
As you have probably noticed I did some work recently to capture parts lists from LEGO's customer services pages, and display them on the set details pages.

As a result, the master the parts list has some 26,000 entries, and after combining it with lists from LEGO, the colour and type of around 23,000 of these is known and is shown in the lists [1]

So, there are 3000-odd that we don't know the colour of. I could look at each image myself and enter it but that would take ages and would soon get boring. I am therefore going to try 'crowd sourcing' the data, and that's where you come in.

I'd like you to visit the part identification page at http://www.brickset.com/parts/datacapture/ and follow the instructions. Then repeat 10, 20, 1000 times, until you get bored.

Every part is shown to two people. If they both agree on the colour then it's recorded in the database. If they don't then the process starts again for that part.

Not all colours are shown in the list, just the common ones. This is because identifying some of the silvers, in particular, and light pastel colours is tricky. So, there's a 'None of the above / I am not sure' option and if two people can't identify it then a 'panel of experts' (me, Martin and bluemoose) can take a look and try and assign the right colour.

The page has been trialled with some guinea-pigs and it seems to work quite well.

If it's a total balls-up, parts whose colour is recorded in this way are marked so can be wiped if necessary without affecting the integrity of the rest of the data.

Also, the system records who enters what in a separate table and I will use this to see who has made the most correct entries and will award a prize once it's finished (not sure what yet, I'll come up with something!)

So, please give it a go, and let's see how quickly we can do it!


[1] tomorrow, I'll be rolling out some more parts-browsing pages which will make use of this information. The ultimate aim is to rival BrickLink's inventories but that's a long way off yet!

Comments

  • CCCCCC UKMember Posts: 17,788
    edited November 2012
    What are you doing about things like this.

    image

    It is claimed to be a part, when really it is a collection of parts. Do you want torso colour in this case?

    It also happens with whole Duplo figures. Torso or legs in that case?
  • BrickbaseBrickbase Member Posts: 100
    Yes, I was also wondering about it. Just picked the color which was the most visible and obvious. Already did some color picking, will do some more later tonight.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,440
    I believe LEGO uses the torso colour in such cases. It could be argued that the is no wrong answer, if 2 people agree, it must be right :-)
  • DadsAFOLDadsAFOL USAMember Posts: 616
    You should add more discontinued colors... This one is aqua (or whatever the Lego color name is) and I couldn't identify it for you.
    image
  • DadsAFOLDadsAFOL USAMember Posts: 616
    Also we need flesh (or whatever the Lego name is) in the list of current colors for minifig heads and friends torsos.
  • DadsAFOLDadsAFOL USAMember Posts: 616
    ...and Medium Azure...

    image
  • davee123davee123 USAMember Posts: 808
    Maersk blue!

    DaveE
  • caperberrycaperberry LondonMember Posts: 2,226
    Re colours not included...
    Huw said:


    Not all colours are shown in the list, just the common ones. This is because identifying some of the silvers, in particular, and light pastel colours is tricky. So, there's a 'None of the above / I am not sure' option and if two people can't identify it then a 'panel of experts' (me, Martin and bluemoose) can take a look and try and assign the right colour.

  • DadsAFOLDadsAFOL USAMember Posts: 616
    ^ yeah, got that. just trying to make it a little easier on them.
  • davee123davee123 USAMember Posts: 808
    Another tidbit-- I got a part that I would call "lime", and I see that there's a "light yellowish green" option, but I don't know if that's the color I'm thinking of. You could either:

    1) Provide the BrickLink/LDraw/Peeron/LEGO Color name for cross-reference
    2) Provide some sample sets that the given color came in, that the user might be familiar with.

    DaveE
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,440
    I'm deliberately avoiding using BrickLink names although I agree it causes confusion.

    You can see the sets the part comes in using the link below the image and thus get to the inventory at BrickLink to check if needed.
  • JosephJoseph Member Posts: 651
    edited November 2012
    A couple times I've gotten this part (http://cache.lego.com/media/bricks/5/2/4111943.jpg) but, looking a the BL parts list, it says (both times) that the part was only used in black in that set. Is this a faulty inventory?
  • JosephJoseph Member Posts: 651
    edited November 2012
    Also, is every part shown to two different people or just two times? I've been given the same part (from the same sets) twice several times.
  • davee123davee123 USAMember Posts: 808
    Right, and I actually confirmed at BrickLink that the color was "Lime", but... BrickSet only has one "lime-ish" looking color, whereas BrickLink has Lime, Bright Green, Medium Green, Olive Green, and Medium Lime. I'm basically asking "How do I know what color BrickSet is talking about when it says 'Bright Yellowish Green'?"

    If that color had a little popup, which showed some other known examples of elements in that color, and what sets they came in, I could probably tell which one you're referring to. But otherwise, although I can tell the color the part you're asking about, I can't tell which of your names I should associate it with.

    DaveE
  • rocaorocao Administrator Posts: 4,288
    If we quickly realize that a lot of people are donating their time, should we change the required agreement to something higher than two people?
  • JosephJoseph Member Posts: 651
    ^^ Actually, I wasn't complaining about the color names (although that is a bit problematic) but rather the fact that it gave me a picture of a chrome/silver part when the set only had it in black.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,440
    edited November 2012
    ^ I think that probably is black, it can be hard to tell sometimes, particularly on that sort of part.

    Good idea DaveE, and it might be as well to put BL's colour name in brackets in the tool-tips, too.

    Overnight, total of 2594 entries have been made to the system and 777 parts have been completed. Pretty good going!

    ^^ I think that will only be necessary if inaccuracies creep in with only two.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,440
    I've uploaded some new pages for browsing parts, starting at

    http://www.brickset.com/parts/browse/

    Still a w.i.p. but you can hopefully see where it's going.

    This page shows all parts that have had their colour defined using the crowd-sourcing, and as you can see it looks as if we have close to 100% accuracy

    http://www.brickset.com/admin/utils/parts/verify/

    Those with colours not in the list are relatively few,

    http://www.brickset.com/admin/utils/parts/verify/?colour=0.

    I can give people 'parts admin' role that enables editing of colours and specifying other colours not in the list currently. DaveE, would you be willing/like to help with that?
  • mressinmressin Lego City... erm LondonMember Posts: 843
    edited November 2012
    rocao said:

    If we quickly realize that a lot of people are donating their time, should we change the required agreement to something higher than two people?

    Huw said:

    ^^ I think that will only be necessary if inaccuracies creep in with only two.

    There is an old creed among seamen regarding navigational instrument redundancy (e.g. additional compasses, chronometers, etc.) which might apply here:

    "Either take three chronometers with you, or just one. Never take two."
    Jenni
  • AanchirAanchir United StatesMember Posts: 2,803
    Oh boy. If you need color help then I'm your guy. But to be honest, identifying the colors from the thumbnails is a painfully vague way of going about things. Here is the Customer Service database template I use for identifying the details of parts. Replace the seven-digit Element ID in the URL with any other Element ID (the number which specifies a color and shape, found in the back of most modern instruction booklets), and the template will tell you the exact color of the part according to TLG's naming scheme. Incidentally, it can tell you this whether or not a picture is available of the part in question.

    Of course, the tricky bit is learning how the TLG names correspond to the Bricklink naming scheme and others used by AFOLs. This is where I happen to have a good deal of expertise.

    Schoolwork is taking up a lot of my spare time this semester, so going through the Brickset database set-by-set seems like something I might not have time for. But in general, feel free to ask me about any of TLG's color names that you are unsure about.

    Here are some pointers to get you started:

    Earth Orange is TLG's name for Classic Brown. Do not confuse it with Bricklink's Earth Orange, which TLG calls Light Orange Brown

    Brick Yellow is TLG's name for Tan.

    Dark Green is TLG's name for Classic Green. Do not confuse it with Bricklink's Dark Green, which TLG calls Earth Green.

    Light Purple is TLG's name for the pink color that debuted in 2004, which Bricklink calls Bright Pink. It replaced Light Reddish Violet, the previous pink color, which Bricklink usually just calls Pink.

    Bright Purple is TLG's name for the dark pink color that debuted in 2004. Bricklink identifies both this and the color it replaced, Medium Reddish Violet, as Dark Pink.

    Silver Metallic is the Pearl Silver color which first began to replace the previous color, Silver, in late 2010. It became standard for all sets in 2011. Bricklink calls it Flat Silver, but it's only one of several colors they give this title.

    Titanium Metallic is the darker silver color which emerged at the same point on the timeline. The color it replaced was called Metallic Dark Grey, and Bricklink calls both colors Pearl Dark Gray.

    Transparent Bright Green is TLG's name for the new Transparent Green that debuted in the Atlantis theme. Bricklink conflates this with a previous, unrelated color, Transparent Bright Yellowish-Green. Their name for both colors is Trans-Bright Green.

    In general, you should rarely trust the Bricklink database about any metallic colors or many transparent colors, because they mislabel things all the time. There are at least three different colors (Red Flip/Flop, Copper, and Reddish Gold) which are obviously different in real life but which they label as Copper without any distinctions.

    Ideally, Brickset would just use the TLG color names and not waste time with converting them into established AFOL color names, since AFOL databases often tend to be wildly inaccurate in how they identify certain colors. But at the same time, I can see how that might make this parts database be less useful to people. In any event, regardless of what naming scheme the site follows, I definitely encourage you to use the hyperlink I provided above to identify if any two parts are actually the same color, rather than just making random guesses.
    rocaoleemcgOldfan
  • leemcgleemcg Member Posts: 607
    I've just gone through a few parts, but having seen @aanchir's post, has this process become irrelevant? Pretty much every part that I got given came up just fine with the link that he gave, giving the colour as per TLG's database.

    Is it possible to just parse the information coming back from this link? At the very least, you should put the contents of that link on the page.
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,440
    I wasn't aware of the URL that @aanchir has posted, so thanks for that. It will indeed be possible to call that page for every part that data is needed for and extract it from the page.

    It'll have to wait until next week now though!
  • davee123davee123 USAMember Posts: 808
    That's a handy link-- also provides the design ID! Now if only it also showed alternate Part Numbers!

    DaveE
  • AanchirAanchir United StatesMember Posts: 2,803
    davee123 said:

    That's a handy link-- also provides the design ID! Now if only it also showed alternate Part Numbers!

    DaveE

    Usually alternate part numbers are either unrelated to anything LEGO keeps track of internally (i.e. fan-made part numbers that show the relationships between similar parts) or just separate part numbers for parts TLG considers separate pieces rather than "alternates" of the same piece. So it'd be hard for them to include those in their database.

    What I do wish they included was the Material ID (color number) for the parts. They include the names of colors but not the numbers, and knowing the numbers would help me make my knowledge of LEGO colors more comprehensive. Just today, while finally trying to make a spreadsheet that describes TLG-Bricklink color name equivalencies, I encountered two colors I had never before heard of: Transparent Light Green (a Clikits color, not to be confused with Transparent Bright Green or Transparent Bright Yellowish Green) and Flamingo Pink. Knowing their numbers would make them easier to put on a chart, as well as easier to cross-reference with other information.
  • AanchirAanchir United StatesMember Posts: 2,803
    On a side note, I look forward to the info being sourced directly from the LEGO parts database, because that will fix errors like we see on some pages. For instance, Turahk's Parts List lists his back, head, and foot elements as Bright Red when they're actually Metallic Bright Red (an easy mistake to make since I don't believe any AFOL databases even acknowledge the existence of Metallic Bright Red-- even in BIONICLE's heyday I used to think the pearliness of these parts was some material/molding quirk and not a deliberate aspect of the color itself). By getting all of this info straight from the database, it will put Brickset ahead of most other LEGO reference sites in many regards and will hopefully make it easier for corrections to be made to other databases.

    One thing that could also be interesting would be if Brickset could list parts for which pictures aren't available at service.LEGO.com, since that would make set inventories a lot more comprehensive. But I don't know if this is possible since the customer service site, at least, won't list pictureless parts in a set's inventory, and I don't know if that's just because they have some filter that stops those from appearing or because non-pictured parts aren't even attributed to these sets in the customer service database until pics are uploaded.
  • davee123davee123 USAMember Posts: 808
    I think the Material ID is actually different than the color number, actually-- I recall back in 2003 working with LEGO's internal database and the color number for the translucent hose in the back of the Millennium Falcon was actually a standard color, even though the material clearly isn't ABS or PC. I assume that means that they have a separate field for the material somewhere, but that's just a guess.

    DaveE
  • AanchirAanchir United StatesMember Posts: 2,803
    davee123 said:

    I think the Material ID is actually different than the color number, actually-- I recall back in 2003 working with LEGO's internal database and the color number for the translucent hose in the back of the Millennium Falcon was actually a standard color, even though the material clearly isn't ABS or PC. I assume that means that they have a separate field for the material somewhere, but that's just a guess.

    DaveE

    "Material ID" is just the term I am familiar with from LEGO Digital Designer (and my dealings with the LDD programming team through Eurobricks's LDD moderator Superkalle), in which it and the color number are one and the same regardless of the type of plastic. It's possible that even if the actual Material IDs specify the type of plastic as well as the color, they are still based on the color number. I have seen some color charts with translucent colors labeled as (for instance) PP182, which would a bit more specifically refer to a Transparent Bright Orange part molded in polypropylene, like the translucent blended tips of some BIONICLE weapons. It should also not be ignored that the material a part is designed to be molded in is often covered by the Design ID (part number)-- for instance, the rubbery minifig spears from 2011 Hero Factory sets have a different Design ID than the ABS minifigure spears used in most other sets, despite having almost exactly the same shape. I wouldn't be surprised if molds in general have to be specialized according to what type of plastic they are going to be molding.

    Regardless, whenever I say "Material ID", you can assume that I am referring to the color number. The only reason I really prefer that terminology is because it is consistent with the terms Design ID (part number) and Element ID (what Bricklink would call a "part color code").
  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,440
    @aanchir,

    > Transparent Light Green and Flamingo Pink.

    These do not appear on official lists I have that provide information on parts in production since 2006.

    You are right about Turahk's Parts List: the colour of those parts was ascertained from Bricklink's part and colour code data (http://www.bricklink.com/catalogDownload.asp) which I used to fill in some blanks before this exercise. Clearly if they are inaccurate I will ditch them and attempt to gather data for all the parts in the database from the link you provided. If I'm successful I will publish the output here.

  • HuwHuw Brickset Towers, Hampshire, UKAdministrator Posts: 6,440
    edited November 2012
    I've scraped data for some 20000 pieces so I've retired this page for now. Thanks everyone for your help anyway.

    I'll make the list downloadable in due course.
  • AanchirAanchir United StatesMember Posts: 2,803
    Huw said:

    @aanchir,

    > Transparent Light Green and Flamingo Pink.

    These do not appear on official lists I have that provide information on parts in production since 2006.

    They are in set 7549-- check it on service.LEGO.com to see for yourself! According to Brickset, this is a 2006 set, although perhaps it was produced earlier and only released in 2006? I certainly wouldn't be surprised if the colors were discontinued in 2006, since a lot of colors disappeared from the palette around that time (examples: lots of metallic colors from BIONICLE/Knights' Kingdom and most of the Clikits-specific colors).
  • rocaorocao Administrator Posts: 4,288
    edited November 2012
    Aanchir said:

    Oh boy. If you need color help then I'm your guy.

    This has my vote for understatement of the year.
  • atkinsaratkinsar Member Posts: 4,272
    Seconded!

    @Huw, next up, crowd sourcing missing database info ;-)
  • CCCCCC UKMember Posts: 17,788
    rocao said:

    Aanchir said:

    Oh boy. If you need color help then I'm your guy.

    This has my vote for understatement of the year.
    We know where the mystery prize should go.

Sign In or Register to comment.
Recent discussions Categories Privacy Policy