Lego Classification Project

stuartmacfarlane

Hi Guys,

I'm currently building a Lego Classification website that will initially allow you to upload a picture of any of the 10K+ minifigures that exist in the Lego brand and it will correctly identify which one and provide you links to bricklink and other partner sites.

Eventually this project will expand to provide the same services for ALL Lego parts!

Completely free of charge service and not planning now or in the future to charge for this as it is just something cool I'm working on.

I do however need help from the community as to train my machine learning scripts I need at least 100 different pictures of each minifigure to get the accuracy level good enough to push something live.

If anyone is interested in helping out I'm putting together a mobile ready website that will allow you to upload pictures of your various minifigures and classify them under the correct name.

In exchange every contributor will get early access to the project and be able to help develop/contribute further towards it if they wish.

Drop a reply to this thread if you would love to help and I'll update this post with more specific details as soon as I'm ready in the coming weeks :)

Stuart

stuartmacfarlane

Reserved for Q&A

Q. Why don't you just use Google images?
A. I would need to seek permission from each image owner and that would take way longer than involving the community. I also need as many diverse images as possible of each minifigure.

Astrobricks

Is the angle the picture is taken from important? I know in facial recognition this is an issue.

stuartmacfarlane

Astrobricks said:

Is the angle the picture is taken from important? I know in facial recognition this is an issue.

Not at all in fact that's the very reason I need community help as lots of diverse photos make it even more accurate in the case of my code.

Different lighting, scenes and even black and white images help :)

The only key thing is that the object is clearly the focus point in the photo and ideally no other minifigures in it except the one.

Fizyx

Sounds like an awesome project that would make the 'identify this LEGO piece' thread go the way of the dodo.  I am wondering if you have a method for accounting for the presence/absence of accessories, including weapons and headgear, etc.

EDIT:  I am also totally willing to help out.  I really like the idea, and it would make sorting bulk used purchases so much easier.

stuartmacfarlane

Fizyx said:

Sounds like an awesome project that would make the 'identify this LEGO piece' thread go the way of the dodo.  I am wondering if you have a method for accounting for the presence/absence of accessories, including weapons and headgear, etc.

EDIT:  I am also totally willing to help out.  I really like the idea, and it would make sorting bulk used purchases so much easier.

It is indeed fun and I've already done a proof of concept using 2 minifigures from series 9 as a test with near perfect results :)

That is a great idea but not something I can fit in right now, the code I'm using should however be able to easily do that with some additional training to look for specific things on each minifigure. I'd probably look at doing this after the initial minifigure classification is done and fully tested.

Keep an eye on this thread and I'll be sure to update everyone as soon as I have the image sharing side of the website online.

Pebbles

It sounds very interesting, and if it works it will be a very helpful resource. :D

CCC

A tool that identifies minifig parts is probably even more useful. The problem with identifying official minifigures is what happens when someone switches a head or the legs or the hair? It is no longer an official minifigure and so doesn't exist.

The torsos are not that bad, goatleg can quite easily narrow down the search if you know what keywords to use for bricklink. Heads are the problem as many have very minor differences from similar ones. But a photographic version of that would be great.

stuartmacfarlane

CCC said:

A tool that identifies minifig parts is probably even more useful. The problem with identifying official minifigures is what happens when someone switches a head or the legs or the hair? It is no longer an official minifigure and so doesn't exist.

The torsos are not that bad, goatleg can quite easily narrow down the search if you know what keywords to use for bricklink. Heads are the problem as many have very minor differences from similar ones. But a photographic version of that would be great.

Yeah I can see how that might be a problem. The software I'm using is quite advanced machine learning software that goes well beyond just image comparison and rather builds a picture of similarities in each of the training images supplied, much like a human brain does when it looks at an object and recognises what it is based on shape, color, location of object etc.

I think I can expand this project later to include some sort of minifigure breakdown so it would identify the part numbers for the head, torso, legs and accessories on each image submission, added that to my little roadmap :)

drdavewatford

Interesting project. I have no idea how feasible it is, but I hope you succeed!

I agree with @CCC that a tool that identifies minifig parts would be more useful than one that identifies whole minifigs - manually searching through Bricklink to identify torsos or head prints gets old very fast....

stuartmacfarlane

drdavewatford said:

Interesting project. I have no idea how feasible it is, but I hope you succeed!

I agree with @CCC that a tool that identifies minifig parts would be more useful than one that identifies whole minifigs - manually searching through Bricklink to identify torsos or head prints gets old very fast....

Ok seems like this is a common theme/problem you guys are having so what I might do is revise my plan and instead of classifying the entire minifigure I may just classify all the heads, torsos, legs and accessories into their own searchable networks and then that will at least give the project some legs (no pun intended).

Later I can see if I can get it to take a minifigure as an input and then tell you what head, torso, legs and accessories were automatically found.

I've actually reached out to Jaclyn and Russell @ Bricklink and asked if they can grant me API access as that will help massively speed up my development and allow me to return detailed information on each search + link back to their website.

Fauch

I think some months, or maybe years ago, someone talked about a very similar project here. maybe you? someone remembers better?

datsunrobbie

Being able to identify individual parts would be a major help, since most of my minifigures are not assembled "correctly". 

stuartmacfarlane

Fauch said:

I think some months, or maybe years ago, someone talked about a very similar project here. maybe you? someone remembers better?

Wasn't me I'm afraid however if someone else has been doing something similar past or present I'd love to cross-share information :)

Fauch

that sounds similar to this : http://bricksetforum.com/discussion/comment/461538#Comment_461538 ?

stuartmacfarlane

Fauch said:

that sounds similar to this : http://bricksetforum.com/discussion/comment/461538#Comment_461538 ?

Funnily enough his videos along with my passion for Lego is what made me start the project in the first place :)

Couple of very big differences though on the one I'm developing:

1. Will be usable via the web and maybe mobile in the future.
2. Uploaded photos of parts don't need to be professional or even on a white background.

Here is a photo I've just run through my test platform and it was able to correctly identify it as minifigure part no. col130 aka Cylopse from series 9.

What you see below is the exact same size and quality image I gave the software.

Priisholm

Hey i would love to help you in thies project. I also have quiet a few exlusice minifigs from Mearsk and i have Jørgen Viig and on other Lego chief minifig. So i think that i can help with some picturs.

stuartmacfarlane

Priisholm said:

Hey i would love to help you in thies project. I also have quiet a few exlusice minifigs from Mearsk and i have Jørgen Viig and on other Lego chief minifig. So i think that i can help with some picturs.

Thanks I'll be in touch with everyone soon once I have a solid method for collecting peoples images :)