Shopping at LEGO or Amazon?
Please use our links: LEGO.comAmazon
As an Amazon Associate we earn from qualifying purchases.

Automated BrickLink cost minimization program available



  • Tevans333Tevans333 Member Posts: 152
    Can anyone leave some step by step instructions for Octave?
  • BrunoBenBrunoBen Member Posts: 1
    I have a trouble with ascii but here's my way to install Octave
    Can anyone leave some step by step instructions for Octave?
    Sorry for my broken english as it's not my native language

    Please download octave here Windows binaries/Octave 3.6.1 for Windows MinGW installer/
    Choose Octave3.6.1_gcc4.6.2_20120303.7z
    Decompress with 7zip or winrar at root
    Ex: C:/Octave

    Then download octave gui here
    Choose GUIOctave v1.6.0 [7.86Mb]
    Launch octavegui
    Give it where octave.exe is
    For instance c:\Octave\Bin\Octave.exe

    You have ready in task bar of GuiOctave

    Decompress latest version of scripts in c:\octave\apps for instance

    Choose in adress bar the folder

    Type RUN_ALL
  • AvengerDrAvengerDr Member Posts: 453
    hey i just tried your last version and with the default .bsx file it gives me the following error:
    Reading wanted list Wanted List for 30053 mini Republic Cruiser.bsx
    Requesting data for 1 of 23: Light Bluish Gray Bracket 1 x 2 - 1 x 4
    Error using >=
    Matrix dimensions must agree.
    The only change I made was to change US to EU. This is being run from Matlab r2011b.
  • Bosstone100Bosstone100 Member Posts: 1,431
    I ran the scripts on a modified .bsx and it looks like it finished. I looked at the Minimum Cost Search Results.txt file and only see the results of the original sample.

    Where does the output go exactly?
  • graphitegraphite Member Posts: 3,275
    ^ did you update the input file name in the "parse_wanted_list_and_price_guide.m" file? There is a line in there to change to the name of your input *.bsx file. If you already had that changed try deleting the 2 *.mat files and the minimum cost *.txt file that are the sample outputs.

    And the final output should go to a *.txt file as well as it should print in the Octave window.
  • atxdadatxdad Member Posts: 68
    For those following this development, I've recently made the most significant update yet: support for multi-core CPU parallel processing for drastically reduced run times.

    As noted earlier, Matlab does not inherently exploit multiple CPU cores which are so common these days, but in my tests, simultaneous Matlab sessions do seem to run on separate cores. Therefore, I automated the division of a large search into smaller jobs that each run in its own Matlab session, and when complete, all results are concatenated into one final report. I can see total CPU utilization of 100% with this modification, and testors confirmed it works with Octave too.

    Lastly, for completeness, the search can now support the inclusion of LEGO Pick-A-Brick as a candidate vendor. It requires some manual data entry to use, but it's there now for those seeking a true least-cost solution.

    Thanks for the continued suggestions and especially the code samples for me to copy!
  • Bosstone100Bosstone100 Member Posts: 1,431
    @atxdad - do we have to split up the .bsx into four files or will all session work together to find the final result?
  • atxdadatxdad Member Posts: 68
    Nope. Just make one wanted list. The program divides the search labor among as many cores as you wish to throw at the problem. I tested up to 8, but it shouldn't have any limit.
  • graphitegraphite Member Posts: 3,275
    It is a very nice improvement. The 4 vendor searches that I've done that would normally take probably 6-8 hours or more take about 3 running on 7 cores. Would be less probably but I think atxdad is still working on balancing the work when it is split up.
  • effalconeffalcon Member Posts: 71
    very nice update.
  • CoolsplashCoolsplash Member Posts: 935
    Can anyone help me walk through the installation for this script/program?
  • Tevans333Tevans333 Member Posts: 152
    I'm having trouble getting multicores to run in Octave.

    The script prompts me the following:

    Multi_core CPU processing enabled. Instructions:
    1) Start 2 additional Matlab session(s) if not already running
    2) Ensure all working directories are hte same as this sessions
    3) Run the following 2 command(s) in the other 2 session(s)
    In session 1 run: find_minimum_cost_remote(1)
    In session 2 run: find_minimum_cost_remote(2)

    I'm good up until step 3. Whenever I try to run the command it tells me the script doesn't exist. Can someone please let me know how the command should look? I'm typing "run find_minimum_cost_remote(1)"
  • atxdadatxdad Member Posts: 68
    @Tevans333 - Although I use Matlab, here are some things to try; Octave users can correct me if these aren't valid commands:

    1) Verify that the files 'find_minimum_cost.m' and 'find_minimum_cost_remote.m'
    both exist in the same directory

    2) In the two additional Octave sessions, do this:
    a) Use the 'cd' command to point the workspace to the directory verified above
    b) Use the 'dir' or 'ls' command to verify the scripts are in your current working directory
    c) Run these commands:
    which find_minimum_cost.m
    which find_minimum_cost_remote.m
    They should both list the current working directory

    If any of those verifications fail, PM me the screen messages and we'll try some more.
    Maybe an Octave user can confirm that you have launched the functions (not scripts) with the correct syntax.
  • graphitegraphite Member Posts: 3,275
    I'm not sure why but for some reason you don't have to type 'run ' in front of the find_minimum_cost_remote(x) calls in Octave. That is probably what is causing it not to work for @Tevans333. I think it has to do with it being a "function" or "script" instead of a "program" but @atxdad would know the difference between 'find_minimum_cost' and 'find_minimum_cost_remote' from a programming standpoint not functionality.

  • Tevans333Tevans333 Member Posts: 152
    Thanks guys for the quick reposnes.

    @graphite, you are right, I was typing run and I didn't need to be. Now I'm happily running @atxdad's scripts on multi cores:)
  • Tevans333Tevans333 Member Posts: 152
    Ok, next issue I could use some help on:)

    After running it on 4 cores it completed a 4 vendor, 33 lot search in about 2.5 hours, very happy with this! But at the end it did not save to the results to the "Minimum cost search results.txt" file. The script prompted me to wait untill all sessions had completed. I did this and made sure each had a command prompt. I then hit a key to continue and have the results saved, only problem is the results file only has this:

    Checking 1.06133e+009 combinations of 4 vendors for those that have desired quantities...

    Search completed for this session. Inspect progress of other Matlab sessions
    and press any key to continue only when other results are ready. PAUSED
    Loading results: multiporc_output_1.mat
    Loading results: multiporc_output_2.mat
    Loading results: multiporc_output_3.mat

    I made sure each session was working in the same directory. Is there a way to generate the results without running the 2.5 hour search again? What may have gone wrong?
  • atxdadatxdad Member Posts: 68
    @Tevans333 - The likely answer is that there were no solutions found, and with nothing to report, the program just stops. I've since added some debug text to make that clear, i.e. "This page intentionally left blank." I'll PM you with a few more questions soon to verify this is what happened.

    This is probably a good time to say that any run-time questions like this can be sent to me directly. Remote debugging is sometimes an iterative process, and I'm guessing few would be interested in my whole conversation.
  • caperberrycaperberry Member Posts: 2,226
    edited May 2012
    Hi guys, I wanted to ask a big favour - would anyone be prepared to run a report for me in this program? I don't have Matlab and need to cost up a model for a charity. It has approx 130 lots.

    I can supply the parts list as BrickLink XML, .bsx, .ldr or .txt

    Any help gratefully received!

  • caperberrycaperberry Member Posts: 2,226
    edited May 2012
    oops double post
  • atxdadatxdad Member Posts: 68
    Sure, @caperberry, I'll run it for you. I'll PM you with details of what I need.
  • Mad_DogMad_Dog Member Posts: 71
    I have been using the program for a few weeks and for me works quite good, I am open to any improvements on it.

    Running list of 115 batches, 3091 pieces, in 2 different files of 37 batches / 837 pieces and 78 / 2254 goes more or less OK with default options, of course 4 shops is looooong calculation, and not always successful, other searches are much better, so keep low batches and merge after, that is the best option.

    Atxdad, I am happy to keep checking any new version you may need, keep the good work.


  • caperberrycaperberry Member Posts: 2,226
    edited May 2012
    OK I can report back that this is great. In short, I'm saving about £15-20 on an order that I reckon would have cost £65-70 had I just used my normal route of trying to do some manual analysis of the BL Wanted by Store results.

    The parts in my order were not expensive but I had 140 different lots, which meant I had to buy from at least 3 sellers. But it turned out buying from 4 vendors was cheaper, even after adding (estimated) postage manually (which of course cannot be calculated automatically).

    As the program cannot limit by country/territory, I went through the many output options provided and rejected non-European stores; there were still several options for me to choose from.

    The program helped me realise I would have to compromise in some way, which I did by having some Used parts instead. Not an easy decision, but great to be able to make it fully informed.

    Now using BrickLink the usual way is just depressing! - so much effort just to know that you are getting a decent price, and it feels like BL is heavily weighted in favour of the seller. A program like this is the way forward and I hope @atxdad has continued success developing this and getting it out there. If BL wants to stay successful they'd be wise to integrate this program into the site, before someone else just does it elsewhere...
  • Bosstone100Bosstone100 Member Posts: 1,431
    It really is an amazing program.
  • Tevans333Tevans333 Member Posts: 152
    BEWARE!!! This program is costing me a ton more money than before I started using it...because it makes projects so much easier and cheaper I find myself doing more projects:)
  • caperberrycaperberry Member Posts: 2,226
  • MolicanMolican Member Posts: 29
    Can anyone send me GUIOctave v1.6.0 because the site is down since at least some days? PM me for my email.
  • samiam391samiam391 Member Posts: 4,506
    The program doesn't seem to be working for me. Every time I click "StartLDDManager" nothing happens at all. I've installed the Access Runtime, so that's not the problem. Any suggestions/tips? Hope I'm on topic here... Right now, I really want any sort of automated system that will upload my parts to BL, so if there's something better than LDDmanager, please let me know!
  • graphitegraphite Member Posts: 3,275
    did you read the readme.txt? Specifically "For Access 2007 and 2010 users - for LDDManager to work you may need to add it to "Trusted Locations".
    Read here how to do that:"
  • pilfospilfos Member Posts: 4

    I've changed the numproc value to 8 in find_minimum_cost.m but I don't know how to open new sessions of Octave.

    And now, I'm receiving this error:

    >> RUN_ALL
    Reading wanted list 10179-bb.bsx
    Requesting data for 1 of 41: Light Bluish Gray Technic, Pin Connector Round (Pin Joiner Round)
    warning: E:\Dev\Octave\apps\readpriceguide.m: possible Matlab-style short-circuit operator at line 63, column 32
    error: findnextstring: mx_el_ge: nonconformant arguments (op1 is 1x41, op2 is 0x0)
    error: evaluating argument list element number 1
    error: evaluating argument list element number 1
    error: called from:
    error: E:\Dev\Octave\apps\findnextstring.m at line 7, column 6
    error: E:\Dev\Octave\apps\parsepriceguidestring.m at line 33, column 14
    error: E:\Dev\Octave\apps\readpriceguide.m at line 52, column 92
    error: E:\Dev\Octave\apps\parse_wanted_list_and_price_guide.m at line 80, column 58
    error: E:\Dev\Octave\apps\RUN_ALL.m at line 13, column 1

  • atxdadatxdad Member Posts: 68
    @pilfos It's my experience so far that almost all run-time errors in the parsing scripts are due to unexpected currency types. If you are using the scripts outside of the US, check the notes for international users in the README, otherwise send me a PM and we'll try to debug further.
  • graphitegraphite Member Posts: 3,275
    As far as opening multiple instances of Octave, first do you have a multi-processor/multi-core system? As far as windows goes, just run the program (click the shortcut) for as many you want to open. Once you have the 8 open you need to open the Task Manager (right click in the open space on the start menu/task bar and click 'Start Task Manager'). Sort by process name, you should see 8 octave.exe's listed. Right click on each one, go to Set Affinity, and select a different core for each one. Then continue to follow atxdad's directions for running multiple processes.
  • pilfospilfos Member Posts: 4
    yes, the needed files were modified for taking into account the EUR, and the script went fine a week ago. But yesterday, when I was trying to check the multicore option, I started to get this error.

    I have an Intel i7, so I think I can use 8 cores, or only 4?

    But now Bricklink is down, so I can't test your suggestions.

  • dustinmrobertsdustinmroberts Member Posts: 35
    Anybody have any luck using this after all the Bricklink problems? I tried to run a file through and all that was returned in the Minimum cost search results file was this:

    Checking 724 combinations of 1 vendors for those that have desired quantities...
    Checking 261726 combinations of 2 vendors for those that have desired quantities...
    Checking 62988724 combinations of 3 vendors for those that have desired quantities...

    I've run it a few times changing the maximum number of stores each time and still the same results.
  • atxdadatxdad Member Posts: 68
    Knowing of BrickLink's changes, I've been checking functionality of the parser every few days, and it seems to still be working correctly as of now. I'll keep an eye on it during this transition period.

    @dustinmroberts Did the program run to completion when it created that text file output, and are you using the latest version ( In older versions, that output file would mean no solutions were found, so it had nothing to report. More recent versions explicitly say 'No combinations found' to avoid confusion. PM me with any replies and we'll figure it out.
  • kufkuf Member Posts: 66
    I have a trouble with ascii but here's my way to install Octave
    Can anyone leave some step by step instructions for Octave?
    Sorry for my broken english as it's not my native language

    Please download octave here Windows binaries/Octave 3.6.1 for Windows MinGW installer/
    Choose Octave3.6.1_gcc4.6.2_20120303.7z
    Decompress with 7zip or winrar at root
    Ex: C:/Octave

    Then download octave gui here
    Choose GUIOctave v1.6.0 [7.86Mb]
    Launch octavegui
    Give it where octave.exe is
    For instance c:\Octave\Bin\Octave.exe

    You have ready in task bar of GuiOctave

    Decompress latest version of scripts in c:\octave\apps for instance

    Choose in adress bar the folder

    Type RUN_ALL
    Just so you know, the site has been in "Maintenance Mode" since at least May 20th, from what I read searching around. You'll need to use this link:
  • kufkuf Member Posts: 66
    Has anybody made a master file of all bricks available from LEGO Pick a Brick? (longshot, I know)
  • DadsAFOLDadsAFOL Member Posts: 617
    Has anybody made a master file of all bricks available from LEGO Pick a Brick? (longshot, I know)
    I started, but man is it a pain. I got about 10% done and it took an hour. I don't have 9 more hours to give to it.

    I would be willing to pay someone $25 for the PaB bsx file. So all you entrepreneurs, listen up!

  • kufkuf Member Posts: 66
    I was thinking about filtering by color then copying all items over to excel and trying to do a VLookup with descriptions somehow. It's too bad the descriptions don't have the item numbers in them.
    My other idea was contacting TLG and asking them for an excel file of the info, explaining we wanted it so we could consider choosing PAB as a vendor option.
  • brickmaticbrickmatic Member Posts: 1,071
    ^ and ^^ So you guys know what the PaB bricks are, you just need to put it into a bsx file?
  • maquesmaques Member Posts: 96
    I think this could be done very easily* [and I will make one eventually...]

    I've checked PAB already, the item's could be extracted easily from the site, eg:
    and then query all brick details by the appropriate query, eg.:

    *Problems/issues encountered so far:

    - This is not "_A_ shop", so price and currency is different from the "cat=US" vs "cat=DE" (and so on).

    - meanwhile basic bricks match the BL's ID's, like

    LEGO: Element ID: 4211050 - 1 1/2 M Connecting Bush
    Design ID: 32002
    Exact Color: DARK STONE GREY

    would "translate" to -->

    Dark Bluish Gray Technic, Pin 3/4


    LEGO: Element ID: 4275585 - Mini Upper Part "No. 1197"
    Design ID: 76382
    Exact Color: WHITE

    cannot be automatically translated to:

    White Torso Town Prisoner Number 50380, Dark Bluish Gray Stripes, Buttons, Hairy Chest Pattern / White Arms / Yellow Hands

    This applies to most of the minifig parts I guess...

    - I would need a color translation table, which maps LEGO color names to BL color ID-s. This is also not a big task to make, but I haven't completed that yet. If anyone has this already, post it or pm me. Currently available PAB colors [at some I put the BL color ID]... Fill the gap :-)

    #BL_color_ID color_family "exact color [with spaces]"
    11 Black BLACK
    ? Blue BRIGHT BLUE
    ? Blue EARTH BLUE
    ? Blue MEDIUM BLUE
    ? Blue TR. BLUE
    ? Blue TR. LIGHT BLUE
    ? Green BRIGHT GREEN
    ? Green DARK GREEN
    ? Green TR. FLUORE. GREEN
    ? Green TR. GREEN
    ? Orange-brown BRIGHT ORANGE
    ? Orange-brown DARK ORANGE
    ? Orange-brown REDDISH BROWN
    ? Orange-brown TR. BROWN
    ? Purple BRIGHT PURPLE
    ? Purple LIGHT PURPLE
    ? Purple MEDIUM LILAC
    ? Red BRIGHT RED
    ? Red NEW DARK RED
    ? Red TR. RED
    1 White WHITE
    3 Yellow BRICK YELLOW
    ? Yellow BRIGHT YELLOW
    2 Yellow SAND YELLOW
    ? Yellow TR. YELLOW

    - Decide what to do with "unavailable" items. Eg. item exists, but no stock.

    - Updates...
  • maquesmaques Member Posts: 96
    Thread seems to be died... Anyone still interested?

    I did some work on it, beta results here:

    pablist-us.txt has 1532 entries, most of them matched easily with BrickLink, though I added some manual item/group translation tables.

    Color translation table is there too, could be some errors in both the item and the color table.

    80 items left on "todo" list to be manually looked up on BL - 99% minifig parts.

    Could look up other country prices (UK, DE, HU, PL ...)

    Could look up BL prices [1/lots for US/EU etc sellers]

    Could create csv or any different format of the result
  • atxdadatxdad Member Posts: 68
    @maques : The script readLEGOPAB.m expects a .BSX format file (text file with markups), but it only parses out a few of the fields that normally exist in a .BSX. The critical information it needs (part number, color, price) are found in your pablist-us.txt.

    I can write a script that will create a .bsx text file, putting the expected markups around the values you've pulled. I can guarantee to make it readable by readLEGOPAB.m, but it may take some extra work to get it to import to BrickStore if one wanted to visually verify its contents or manually make an edit.

    Is pablist-us.txt the format of your final product, and is there any opportunity for revision? Also note that BL prices will not be necessary, nor will a .csv format. PM me and we can finalize everything. Thanks for the help.
  • DadsAFOLDadsAFOL Member Posts: 617
    @maques The bsx file you created is excellent. There were just a few items that came up as "undertermined" and needed the "b" added to the part number. That was easily done in BrickStore. The handful of remaining items that didn't link can be added manually in BrickStore.

    Thanks for doing this!
  • maquesmaques Member Posts: 96
    Can you let me know what are the items needed to be modified?
    So in case PAB item updates or for any other reason I shall create a new bsx then they would be corrected and wouldn't need to have modified by "end-user" :-)
  • DadsAFOLDadsAFOL Member Posts: 617
    The unidentified were 1x1, 1x2, 2x2 tiles, 1x2 mod tile with grille, and a couple others. In BrickStore, just type "undertermined" in the filter list at the top and you will see them. I didn't go through the 80 unmatched yet.
  • atxdadatxdad Member Posts: 68
    With the PAB inventory importation now getting some attention, an error in my scripts during the reporting of PAB information was found which could cause some runtime confusion but not affect end results. Anyone using the PAB feature can PM me to request a hotfix immediately.

    I am planning to release another revision update soon which will include this fix and few others, but I think I will wait until the work of @maques is complete to ensure compatibility. The draft .bsx file he has already provided is a huge timesaver.
  • maquesmaques Member Posts: 96
    I made some adjustments, so now bsx contains:
    - qty 9999 for item on stock and qty 0 for N/A stock item
    - removed "fixme/error" items - so no ~80 warnings with BrickStore
    - All LEGO/PAB information in Remarks field

    Current issues/todo:
    - 1 unknown item stayed with BrickStore, but I don't yet had time to check why, as item 50861 in color 11 [black] exist.
    [Catalog: Parts: Tire & Tread: 50861 - Tire 21mm D. x 6mm City Motorcycle]

    - The missing/to be checked-and-fixed 86 items are in:
    even if people here can lookup and match 1-1 item and post details here (eg what LEGO PAB Element # is the matching BL item #), then it would be helpful

    - The "undetermined" elements come from EXACT element matching, like 3069/3070, where it comes from LEGO/PAB Design ID being 3069/3070.
    Because for 3069b/3070b the PAB Design ID-s 30070/30039. Since there is no way for me to find out wheter a LEGO 3069/3070 is without or with a groove, so type A or B, I can automatch group (or single item).
    After I automatch 3068/3069/3070/2412, there are still around 16 other items stay in undetermined state, eg. with undetermined stud, hinge, handle, ends&attachments, back and hole types.
    What to do with them?

    Anything else?
  • maquesmaques Member Posts: 96
    Was wondering...:

    Wouldn't it be good/useful/interesting to set up a "phantom" Bricklink Store with the PAB items, so could be updated and matched wanted lists against it?
    Then there could be different ones for countries as the PAB USD/EUR/GBP/HUF/PLN/HUF etc prices are static...

    Problem would be that these shops would come out all the time but you wouldn't be able to order as it would be just a reference shop...

    So it would need BL administration + programming to make these hidden / available per request and also not to count in the ~1500x9999 =~ 15 million extra parts...
  • maquesmaques Member Posts: 96
    is with automatch groups 3068/3069/3070/2412 to "b" (with groove/lip)

    Still don't understand the unknown motorcycle tire. BrickStore's database out of sync, or not/can't do lookup to BL?
  • dustinmrobertsdustinmroberts Member Posts: 35
    Is there currently a way to exclude a particular shop? I keep having a shop show up that only has 1-2 parts, but they have a minimum buy that is more expensive than the parts I would need to purchase.
Sign In or Register to comment.

Shopping at or Amazon?

Please use our links: Amazon

Recent discussions Categories Privacy Policy

Howdy, Stranger!

It looks like you're new here. Sign in or register to get started. is a participant in the Amazon Services LLC Associates Program, the, Inc. Associates Program and the Amazon EU Associates Programme, which are affiliate advertising programs designed to provide a means for sites to earn advertising fees by advertising and linking to Amazon.

As an Amazon Associate we earn from qualifying purchases.