Automated BrickLink cost minimization program available

atxdad

@EdmanZA 's mods to Matlab's URL-reading functions could potentially solve ALL the problems international users have been having: no more guessing at foreign currency codes, the ability to limit vendors to specific countries, etc. I have tested his changes in Matlab, and they work without error. However, I am in need of Octave users to test the same, particularly outside the US.

**If you are an Octave user with any previous experience using the scripts and you would like to help test the new international support, please send me a private message.**

Thanks for your continued interest and support.

piratemania7

Hi Everyone, couldnt find the thread if there is one, so I apologize if this is redundant. But does anyone know how I am able to set up a cost minimization program to build a set? I have no idea where to begin...

andhe

http://www.bricksetforum.com/discussion/3662/automated-bricklink-cost-minimization-program-available/p1

Seems to involve some pretty heavy knowledge of programming and such though. Hence, is beyond me.

Poochy

It's not that complicate to get setup and running actually. Now some people have ran into unexpected issues, but @atxdad has been more than willing to help out whenever possible. You actually do not need to know anything about programming, just have to install about 3 programs.

piratemania7

Can you list which ones? Thanks!

Poochy

^ See the other thread, but basically octave (you may want the GUI interface as well), the actual program from atxdad (he'll email it to you), and maybe BrickStore.

piratemania7

Got it! Thanks!

piratemania7

Admins feel free to close.

jo_pix

excuse me,how I can use and where I can download,if it exist, a software to minimizing bricklink costs?
i understood that i must install matlab or something like this,but where i can download the executable that can minimize the costs?
thnx

Joseph

^ I believe atxdad will set you up if you send him a message.

dvsntt

I am receiving an error running the script I received in April with Octave. It appears to go through the parts and request data, but at the step where it beings processing that data, a dialog box with the error you can see here:
http://habervision.net/offsiteimg/octaveerr.jpg

I can't seem to locate that file that is needed. Am investigate further, but if you have a suggestion, please let me hear it. Thanks.

dvsntt

I have successfully used this script and placed my first order in bricklink based on the results. Works great for 25 lot order. It is, however, processing for days to try and fulfill a 200 lot order. Still working on that.

dvsntt

Thanks to the author for writing and providing this great tool!

badbob001

Does this script support reading a wanted list formatted in the same format used to import wanted lists into bricklink? I ask because there is now a way to export bricklink wanted lists in xml format and not require the use of brickforge to manually recreate these lists first.

The following is a browser script that allows the export magic to happen:

BrickLink Wanted List Exporter
http://userscripts.org/scripts/show/167618

To use the script, firefox users need to install the greasemonkey add-on. Chrome users need tampermonkey (what I use). It only exports what is on the current page, so if your wanted list spans multiple pages, you either have to use the "&wantedSize=1000" in the url trick or simply use the 'append' feature of the script.

An example export is this:

<INVENTORY>
<ITEM><ITEMTYPE>M</ITEMTYPE><ITEMID>tnt016</ITEMID><COLOR>0</COLOR><CONDITION>N</CONDITION><MINQTY></MINQTY><WANTEDLISTID>351209</WANTEDLISTID></ITEM>
<ITEM><ITEMTYPE>M</ITEMTYPE><ITEMID>sw413</ITEMID><COLOR>0</COLOR><CONDITION>N</CONDITION><MINQTY></MINQTY><WANTEDLISTID>351209</WANTEDLISTID></ITEM>
<ITEM><ITEMTYPE>M</ITEMTYPE><ITEMID>tnt017</ITEMID><COLOR>0</COLOR><CONDITION>N</CONDITION><MINQTY></MINQTY><WANTEDLISTID>351209</WANTEDLISTID></ITEM>
<ITEM><ITEMTYPE>M</ITEMTYPE><ITEMID>tnt002</ITEMID><COLOR>0</COLOR><CONDITION>N</CONDITION><MINQTY></MINQTY><WANTEDLISTID>351209</WANTEDLISTID></ITEM>
<ITEM><ITEMTYPE>M</ITEMTYPE><ITEMID>tnt012</ITEMID><COLOR>0</COLOR><CONDITION>N</CONDITION><MINQTY></MINQTY><WANTEDLISTID>351209</WANTEDLISTID></ITEM>
<ITEM><ITEMTYPE>M</ITEMTYPE><ITEMID>tnt008</ITEMID><COLOR>0</COLOR><CONDITION>N</CONDITION><MINQTY></MINQTY><WANTEDLISTID>351209</WANTEDLISTID></ITEM>
<ITEM><ITEMTYPE>M</ITEMTYPE><ITEMID>tnt020</ITEMID><COLOR>0</COLOR><CONDITION>N</CONDITION><MINQTY></MINQTY><WANTEDLISTID>351209</WANTEDLISTID></ITEM>
</INVENTORY>

atxdad

My script only reads BrickStore .BSX files (also XML) as input because exporting from BrickLink wasn't possible until now. From the feedback I've seen, I know some will find that new exporting function handy. Maybe the exporting could include an option to write it out in BrickStore XML format for easy editing.

graphite

The Bricklink XML format is importable into Brickstore (including the wanted list export above) so it is easy to just import it and then save in Brickstore format for use with atxdad's script.

badbob001

I'm sorry but I can't find a way in brickstore to import a bricklink formatted xml file. Please show me how.

graphite

Sorry I forgot there are multiple "feature sets" of Brickstore. The default setting does not have all the "Bricklink Seller" features. You have to go to Help->Registration and choose Demo Mode instead of Private Use. It has a 20 minute pop up reminder to register but it is the only way to use the other features since the developer of the program appears to be MIA.

badbob001

Graphite, thanks for the tip. It works!

I'm getting this error. Anyone know what is the cause?

Saving Wanted List and Price Guide data to: Price guide data.mat
Merging all vendor information from raw PriceGuide data.
Would you like to include LEGO Pick-A-Brick as a vendor? Y/[N]: n
error: merge_price_guide_data: A(I,J,...) = X: dimensions mismatch
error: called from:
error:   \\nas\g\bricklink\Minimization_V9\merge_price_guide_data.m at
 line 175, column 42
error:   \\nas\g\bricklink\Minimization_V9\RUN_ALL.m at line 14, column 1
octave:1>

line 175 is: vendorprice(vendorind,partind)=mean(pricelist(1:vendorqty(vendorind,partind)));

column 42 is at the = sign.

badbob001

I found the issue and the error was due to the quantity field being zero. The wanted list exporter puts quantity in field MINQTY instead of the expected QTY field. Quick fix is to simply rename MINQTY to QTY before importing the XML into brickstore. Thanks to atxdad for some hints.

Bosstone100

For the longest time, everything was working fine. I updated to v9 and can no longer get things to work. This is what I'm getting:

warning: fopen: file found in load path
warning: fopen: file found in load path
Requesting data for 1 of 76: Dark Red Brick 1 x 1
warning: E:\Files\Legos\BRICKL~1\readpriceguide.m: possible Matlab-style short-circuit operator at line 32, column 20
warning: E:\Files\Legos\BRICKL~1\readpriceguide.m: possible Matlab-style short-circuit operator at line 35, column 24
warning: E:\Files\Legos\BRICKL~1\readpriceguide.m: possible Matlab-style short-circuit operator at line 76, column 32

After this, each part iteration gives the warning: fopen message again.

Any ideas?

LegobrandonCP

If anyone is willing to help me run a Brickstore file through Matlab, I would really appreciate it. I don't have the knowledge to run the program on my own, plus it doesn't seem to want to work on my computer.

Please PM me and I'll send you the BrickStore file. I would prefer it if it was set to Canadian currency.

capsoft

Hi, I am no mathlab user, but a .net developer and am looking to build this tool myself. I have taken a look at the bricklink api http://api.bricklink.com/pages/api/welcome.page but it is seller oriented and I am a buyer. So I have to wait for those REST api calls to become available or I have to scrape some HTML myself. I have the bsx and/or csv file for the set I would like to build and I can parse it. But is there an url where you can see the price of the piece at various shops?
So that you can put the piece ID in the url and get a list of shops? Because bricklink requires authentication, I cannot parse that html automatically. or I should copy the authentication cookie of my browser and send that with each request. How do you mathlab folks do this?

atxdad

@capsoft Yes, you can get a list of prices by shop (including quantity available, new and used) sorted by price from the Price Guide page for a given part. From BrickLink main page look under Catalog/Price Guide tabs. As an example, here is a URL that will pull prices for part 3039 in color 7 (blue):

http://www.bricklink.com/catalogPG.asp?itemType=P&itemNo=3039&itemSeq=1&colorID=7&v=P&priceGroup=Y&prDec=2

capsoft

Thank you @atxdad but where are the shopnames?

graphite

^ The store ID # is in the link that the little square next to each line item links to. So if you click one it takes you to that item in the store.

capsoft

Thank you @graphite‌ this will get me started!

FrictionPin

I've already worked out 90% of an application like this in VB if you'd like a jump start shoot me a PM @capsoft. My free time dried up a couple months back so I figured I'd wait until I see the API for BL 2.0 (or 3.0 if you follow their versioning system). Some screen scraping and a few web queries through the already provided web interface on the site, it's not the most optimized system but it's something I dabbled with for a while.

It stops short of the doing the actual lowest price and fewest stores compilations. Everything else works and all the data is there, based on an imported wantlist, I just ran out of gas and put it on the back burner until I have more free time. It's been sitting dormant for a few months now.

capsoft

@FrictionPin‌ why don't you publish your code on github? Mine will be up there soon (I hope)

capsoft

My C# Github repository can be forked ofcourse and send pull requests if you like! Looking forward to suggestions and/or help. Perhaps this is all to soon and we just have to wait for the buyer api of bricklink.

Farmer_John

If someone is able to build an self-contained app to do this application, they could probably make some money...if it was reasonably priced. I would definitely consider purchasing it.

I think more and more AFOLs are going to MOC since TLG is continuously increasing the prices and/or removing discounts on their larger sets. I also believe this program could indirectly help hold down TLG prices on the larger sets if it was widely available. Simultaneously, it would also drive more business to BL.

derfinot

Farmer_John said:

If someone is able to build an self-contained app to do this application, they could probably make some money...if it was reasonably priced. I would definitely consider purchasing it.

I think more and more AFOLs are going to MOC since TLG is continuously increasing the prices and/or removing discounts on their larger sets. I also believe this program could indirectly help hold down TLG prices on the larger sets if it was widely available. Simultaneously, it would also drive more business to BL.

I would definitely pay $10-$20

capsoft

Lol, I have published the sourcecode of my console application on github. You can check out the code and/or make contributions with pull requests. it is now a bit limited because I have the input csv file hardcoded in it and the app does not have the logic yet to limit the amount or origin of the stores yet, but there is a nice start. The app on github is free and open for everyone.

emmtwosix

derfinot said:

I would definitely pay $10-$20

It would pay for itself rather quickly. You could probably save that much on your first purchase!

There's definitely a market for an application - I've been looking for something like this for quite some time.

Bobflip

I've played with the Oracle version of this, but would love an application version! I beta test for various commerical soft synth and plugin companies, and would be happy to help test this out.

rocao

We had some discussion a while back about just how public such a tool should be. The fear side is that if it becomes too widely used Bricklink may prevent the scraping of the data because:

1. it taxes their server
2. price optimization results in lower commission for the given order

dp3dp

atxdad I know this thread is super old but are you able to send me this script? Thank You

arthur999

Hi @atxdad !
Could you send me your script? I'm sure that it's a beautiful piece of programming...
Thanks in advance!

CupIsHalfEmpty

atxdad's Matlab scripts have been written into a executable program with a proper GUI called Brick Wizard http://thebrickwizard.com/ . There are also other programs that have arisen with the same basic functionality. However about February this year Bricklink made a database change and all these programs seem to fail during the database inquiry stage, and I haven't seen any updates come out that have fixed this since. 

If anybody does have luck running one of the GUI programs, please let me know. 

DadsAFOL

CupIsHalfEmpty said:

atxdad's Matlab scripts have been written into a executable program with a proper GUI called Brick Wizard http://thebrickwizard.com/ . There are also other programs that have arisen with the same basic functionality. However about February this year Bricklink made a database change and all these programs seem to fail during the database inquiry stage, and I haven't seen any updates come out that have fixed this since. 

If anybody does have luck running one of the GUI programs, please let me know. 

brick wizard works fine for us.

ColoradoBricks

I also have many issues with BrickWizard with failing to get data/catalog/sellers, I have much better luck with Brickficiency.

CupIsHalfEmpty

^ that's the other one, I couldn't recall the name at the time of my first post. I'll have to do some more experimentation. 

atxdad

Even though I haven't updated the Matlab scripts in a long time, I confirmed that as of today the price data parsing still works correctly, so the Matlab version remains an option should the executables be unsuccessful.  One of the parts in the example file had changed part numbers, so I updated that; if that caused a problem for anyone, just delete it from the example parts-wanted list and retry.  Also know that I do still respond to requests for the scripts, preferably sent in a private message to me.