myBrickset Android App

NickF22

This isn't a major issue - but when I go into the Search option for the first time, I get a "simple" numeric keyboard:


But if I switch to ABC, then back to 123, I get a "phone" keyboard:


Ideally, what I would love, is a calculator-style / computer numeric pad-style one, with 123 at the bottom - not sure if that's available in Android though?

PS Can I also add my eternal gratitude for the work you do on this app!

sid3windr

Mine just switches between 2 keyboards...

Re captcha, I guess that's CloudFlare and not Brickset itself asking for the captcha? Might be something Huw can toggle in the Cloudflare control panel.

insmuh

RikTheVeggie said:

Hi @insmuh.

From your profile it appears you're in Turkey.  I've done some testing and that is one of the areas that brickset.com now requires a captcha for (the first time you visit brickset.com you'll need to complete an "I am not a robot" test).

This affects all API calls too, so all apps using it are affected and not just myBrickset.

I have spent many days trying to find a way around this.  Unfortunately I have not yet determined a solution that allows the user to complete the captcha and then have the API calls honour this.

What this means is that I do not currently have a way to allow users from some geographic regions to use Brickset's API, which means they cannot login - or if already logged in, they cannot sync their data or download the latest sets etc.

One workaround is to use a proxy or vpn based in a different country (eg, the UK).  Once you have logged in once myBrickset will remember your information and you won't have to sign in again, so you can reuse the app without having to connect to a proxy or vpn each time - but it won't be able to access any data stored on brickset.com until you do use a proxy or vpn again.

Sorry for the disappointing news.  If I ever figure this out or brickset.com change their captcha policies then I'll provide an update.  In the meantime I'll look at improving the app so that it reports the issue to the user, so you're not left wondering what's happening.

vpn trick worked for login. and all databases updated including the wanted ones. ;)

thanks for the answer and the solutions.

RikTheVeggie

@insmuh Excellent, glad you got it to work :)  Thanks for letting me know.

@sid3windr yup technically Cloudflare but at Brickset's direction :p  I've had a quick chat with Huw before and he wants to keep everything protected atm including the API.  Maybe I'll be able to persuade him differently eventually!

@NickF22 ah, that's my fault.  Looks like I'm asking for a "number" keyboard by default, but "phone" if the input type is switched.  It might be the same for some devices which is why I missed it before.  I forget now which one I wanted, probably "phone" as it's more useful - I'll check my notes and forum posts.  Android is very limited on the keyboard types I can request, different manufactures tend to implement them differently too.  The "best" solution is for me to implement my own but I don't really want to go down that route!

RikTheVeggie

@insmuh if you have the opportunity, can you try the following release please without the VPN:
http://otissoft.co.uk/mybrickset/bin/myBrickset-112-2.6.3.apk [5.0MB]

If you have Lollipop (5.0) or higher (~90% of devices meet this criteria), you will be presented with a CAPTCHA.  Completing it will proceed to myBrickset as usual.  Currently you will be asked to solve the CAPTCHA on each launch of the app, hopefully I can prevent this in the future.

If you do not have Lollipop, then you'll be shown a message that brickset.com cannot be reached.

@NickF22 this release should also fix the keypad from changing types, I settled on displaying the simple "number" keypad as "phone" doesn't provide any benefit and just clutters the display.

NickF22

@RikTheVeggie brilliant service as always! I've confirmed that I now get a consistent "number" keypad.

RikTheVeggie

Thanks Nick :)

RikTheVeggie

@insmuh this version will attempt to bypass the CAPTCHA once you've solved it once, but the final determination of whether you see it or not is out of my control (Cloudflare / brickset.com).

Please LMK if you're able to test this, I have done so by using a localised proxy so it should do but can't guarantee it.

http://otissoft.co.uk/mybrickset/bin/myBrickset-113-2.6.4.apk [5.0MB]

Really didn't expect to be in this situation (with a working solution) given where I was last week, but am extremely glad and relieved to be so!

RikTheVeggie

Please forgive errors & typos in the above post, I must've been half-asleep.  Hopefully it all still makes sense!

Basically.. after solving a CAPTCHA once, you should not need to solve it again the next time the app is launched.  I've tested locally with a VPN so am pretty sure it will work as expected.

insmuh

RikTheVeggie said:

@insmuh this version will attempt to bypass the CAPTCHA once you've solved it once, but the final determination of whether you see it or not is out of my control (Cloudflare / brickset.com).

Please LMK if you're able to test this, I have done so by using a localised proxy so it should do but can't guarantee it.

http://otissoft.co.uk/mybrickset/bin/myBrickset-113-2.6.4.apk [5.0MB]

Really didn't expect to be in this situation (with a working solution) given where I was last week, but am extremely glad and relieved to be so!

yes this version opened the captcha first and then it could login without using VPN thanks rik. awesome work.. 

RikTheVeggie

That's great, thanks for testing!  I'll hopefully push it out as a formal update by the w/e.

RikTheVeggie

I've recently found out the app that myBrickset uses for scanning barcodes has been removed from the Play Store (they don't want to spend the effort updating it to the latest requirements) so I've been spending a while coming up with a replacement.

That is now done.  The latest solution uses Google's own Machine Learning libraries so is hopefully going to remain supported for quite a while.

I've done some testing locally, but I don't have access to many physical devices with different Android versions.  So I'd like to crowdsource some testing if you're up for it :)

If you'd like to help out, please install the latest version, scan a few barcodes (or just https://brickset.com/assets/images/barcode?code=5702014499867 and verify it takes you to 10181) and reply with your Android version and whether it worked or not, and especially LMK if it crashed!

http://otissoft.co.uk/mybrickset/bin/myBrickset-118-2.6.9.apk [20MB]
As you can see, adding barcode functionality unfortunately caused the app to grow from 5MB to 20MB because of the required libraries :(  I'll keep working on trying to improve this.

sid3windr

Didn't the barcode scanner just ask Android for any app with an intent doing barcode? Because it still works fine for me... I'm using the scanner Google Authenticator also recommends...

RikTheVeggie

That's ideally how I would've liked to code it, and how Android was originally envisioned to work (but never did, as Android didn't enforce names for common actions or even recommend any, so developers were free to come up with their own and there's zero compatibility).

What actually happens (from memory, I'm at work and don't have the code in front of me) is that the app asks for any installed apps that understand what the action "com.google.zxing.client.android" is - which is just the ZXing "Barcode Scanner" app and it's derivatives.  If nothing was found, it directed the user to the page for the ZXing "Barcode Scanner" on the play store - which no longer exists (and it looks like the plus versions may not be any longer) [tho' it looks like an Indian developer has taken the open source code and just republished with the same name (but different package id) - https://play.google.com/store/apps/details?id=com.qrcodescanner.barcodescanner along with implementing in app purchases for something].

I'm surprised that scanning barcodes hasn't been baked into the Android framework yet, it's such a common action.

Hmm.. interesting.  Running Google Authenticator I see a UI that looks like a custom one, but the source says "com.google.zxing.client.android.SCAN".  Even after I've uninstalled the Barcode Scanner app locally it still works.  Going through the code some more, if it can't find a scanner it redirects to the user to "market://search?q=pname:com.google.zxing.client.android" (maybe any of those results are all compatible?) and if that fails then it prompts to download an apk directly (https://zxing.googlecode.com/files/BarcodeScanner3.1.apk - which no longer exists).  Oh, I'm looking at code from 2016.  oops!  I'll try to find the latest repo when I get home.

I'll also try and do some playing on a clean install and see what's going on (does it still prompt for a 3rd party app to be installed), but given that I can't rely on any 3rd party tool existing and there being no agreed upon standard for asking for a barcode to be scanned, I may end up sticking with the current solution of hosting scanning directly inside the app (and taking the 15MB hit of introducing the Machine Learning library, until Google support it as part of the framework - which the docs imply should be happening).

RikTheVeggie

Back down to approx 5MB :D  

As before, I'd appreciate if people could give the barcode scanner in here a quick test so I can get some confidence across multiple versions.

http://otissoft.co.uk/mybrickset/bin/myBrickset-119-2.6.10.apk [4.8MB]

(btw @sid3windr - I can't access the code for any versions of Google Authenticator newer than about 2016, but I've done some tests with the latest version on a clean install and it seems to have a built-in scanner.  It showed me it was possible to have a scanner without bloating the app though, which led to me putting some more effort in and getting back to ~ 5MB!).

sid3windr

Nice! :-) I didn't know it had a built in scanner these days, but at least the installed "Barcode scanner" app is still called up by and working for myBrickset... (https://play.google.com/store/apps/details?id=com.google.zxing.client.android)

Another question out of curiosity - I've opted in to the beta testing on the play store, I assume the apk's you're linking here are not delivered via that mechanism?

RikTheVeggie

I'm haven't pushed these to the beta channel, to be honest so few people signed up to it that it wasn't really worth the overhead of managing releases on there.  I can certainly publish it tonight though, shouldn't take me long.

sid3windr

Didn't mean to push you on that - just might be easier than downloading APKs, but I have no idea how much overhead that puts on you vs just SFTP'ing an APK file to your web host :-)

RikTheVeggie

No worries, was able to push it last night pretty quickly.

If anyone else would like to join the beta program, you can sign up here: https://play.google.com/apps/testing/uk.co.otissoft.mybrickset

If there's a beta release then you'll be prompted to install it the same way you would a regular release.  If a regular release comes out that supersedes it, then you'll receive an update prompt for it as usual.

I try to only push stable builds (release candidates) to the beta channel, so it's a pretty low risk endeavour and chance to get new builds early, but I won't say it's completely risk free and you'll never encounter a critical bug - the chance of data loss is extremely low though and if I ever make changes that affect the sync code or have a chance of removing all your data from brickset.com then I'll explicitly say so in here and in the release notes.

VorpalRyu

Signed up for beta testing.

RikTheVeggie

Minor update on the Beta channel - Export wasn't working (all Android versions) and the app was also unable to download a full sync of data from otissoft.co.uk (Android 9 only).

I haven't heard of any issues with the barcode scanner so I'll probably push it with these fixes in a day or two.

Also available as a direct download: http://otissoft.co.uk/mybrickset/bin/myBrickset-120-2.6.11.apk [4.8MB]

VorpalRyu

Barcodes don't read on my phone, but then again, they haven't been readable for a good little while now. I would have reported it, but I rarely used the function, it's usually quicker for me to punch in the set number.

RikTheVeggie

@VorpalRyu
What behaviour are you seeing - the reader cannot detect a barcode, or it detects one but shows no results?
If the first - can you please send me a screenshot (here, PM or mail) as it may be a format I'm not handling.
If the second - can you please try with the attached as that's one that I know should work (it's for #10181).

  

VorpalRyu

Wow, that is some crazy stuff, the barcode on the screen whips up Eiffel Tower almost instantly, but every attempt at reading an actual barcode off a box, a polybag, whether from a product bought locally, from the EU or the US....no results found.

RikTheVeggie

That's likely more down to the lack of barcode information in brickset.com (it's very hard to source them apparently, especially in any way that allows easy importing into the db).  I'm considering ways to let users submit them but need to work out some logistics, have a way to verify submitted barcodes, and ideally get them uploaded back to Brickset.  That project will have to wait a while though, I've got a big rewrite going on atm.

Can you enter some of your sets in brickset.com and see if the barcode for them is shown there (or you can do the same test in myBrickset)?  I'm guessing that even if they have a barcode they won't have the specific one for your set (Brickset only stores at most 1 UPC and 1 EAN, but sets can have multiple due to regional differences).

VorpalRyu

I'm suspecting it might be Google's barcode reader code, that or my camera resolution isn't high enough to function as a barcode reader thanks to Google's coding. When reading the barcode you posted earlier, the white box surrounded the entire barcode, while trying it with actual barcodes on boxes, it doesn't quite surround the barcode from end to end, so I'd suspect it's missing the data at the edges of the barcode, which is why I'm getting nothing but "no results found."

I tried it with all the 4+ SW sets & all the Ninjago Legacy sets we've got, all have barcodes displayed when I view the set data on the app.

RikTheVeggie

@VorpalRyu
can you send me pictures (here, PM or email) of some of the barcodes you can't scan please.  I'll see if there's anything I can do to improve things.

RikTheVeggie

If anyone's feeling adventurous, I have a preview of the next major release to play with.

CAUTION: There is no upgrade path from this version.  You WILL have to uninstall this to go back to the current release, or when the next major release is officially released.  You MAY have to uninstall this to apply any future previews.  It is unsupported, but under heavy development so I'm likely to quickly fix any issues that are raised.

There should be no data loss with this, but I've only been testing with a dummy account.  So far I have not noticed anything wrong, and syncing etc appears to work as expected.  So it should be safe to use with a live account, but you may want to take a backup just in case!

Installing on top of an existing install is fine, and it will migrate any owned sets etc.  Obviously a clean install works too, and owned sets etc will be download from Brickset as usual.

As to what's changed.. I started off refactoring the database layer (M) to using the latest libraries (AndroidX & Room) but I very shortly found myself reworking much of the interface layer (V) and controller layer (C) too.  I'd estimate that >70% of the code has been refactored.

It should feel a lot faster in general, especially when building or updating the db and when performing large queries or retrieving a lot of results.

The UI has been given a lick of Material Design so should hopefully look a bit cleaner and more "modern".  The Grid view & Expandable List view have both been completely overhauled for improved performance and stability.

Lots of code has been refactored to the AndroidX libraries which should mean that it inherits a lot of bug fixes and improvements, as well as simplifying future maintenance.

I've also introduced automatic error & crash detection which I can monitor online, so hopefully it'll allow me to be a bit more proactive when it comes to investigating and fixing issues.

Known bugs - there may be issues when a background sync starts, but syncing from within the app (on startup or by pressing Refresh) should be fine.  I'm working on removing the dependence on having an Account defined in Google Settings but haven't yet completed it.  It won't cause any data loss etc, but the app may crash on you (but this is intermittent and rare - it requires Android to start a background sync whilst the app is open).

If you're feeling brave, give it a spin and let me know your thoughts or if you find any issues:

http://otissoft.co.uk/mybrickset/bin/myBrickset-300-3.0.0_dev.apk [5.4MB]

SteveG

Samsung Galaxy S7 running Andriod v 8.0.0 crashes every time I try to do something. Sor example, click sets, click owned. Crashes out. Just a heads up.

RikTheVeggie

Thanks for trying it out @SteveG!  The good news is you've just tested the online crash reporter for me, so I know that works now :)  It emails me when a crash happens and lets me see into the reason why, which means I can do some quick analysis now even though I'm not at home.

Looks like the crash is related to a recent column rename I implemented - I forgot to double check all the sorting & ordering functionality.  If you change the sort to anything other than "theme" then it should start working.. but I just realised you can't do that as you need to complete a search first to be able to access the relevant settings.  Darn.  A clean install would work but I don't want to force you to do that.

This shouldn't take long to fix so I'll try to do it when I get home from work tonight and push out a new update.

Thanks again for being brave enough to give it a spin!

RikTheVeggie

Let's try this again shall we.. 

As before, please only install this if you're happy with installing alpha software and being on the cutting edge of myBrickset development.

http://otissoft.co.uk/mybrickset/bin/myBrickset-301-3.0.1_dev.apk [5.3MB]

SteveG

I spent 10 years of my life alpha and beta testing software in the gaming industry. My superpower is breaking things that are about to go to gold master ;) 

Downloading the new version shortly. 

SteveG

Installed and synched and working great. Awesome. 

RikTheVeggie

That's great!  Thanks for testing @SteveG :)  

I'll continue development and will hopefully have something a bit more finished next time.  

RikTheVeggie

Development of the 3.x line of myBrickset has reached the point where I'm comfortable releasing it to a wider audience.

There are a lot of people on the Beta channel that may not necessarily want to try out the 3.x release yet, so I'm shipping it as an Alpha.

Alpha status does not imply that the build is unstable, although there is still a very slight chance of data loss (I'm reluctant to say no chance until I've had multiple people report it's fine) and some of the Android libraries being used are still in Alpha status themselves.

• If you have a production release, it should upgrade to 3.0.2 seamlessly.
• If you have an early Alpha release (3.0.0 & 3.0.1), it should upgrade fine but information on "favourites" & "built" will be lost. If you wish to avoid this, export your data first
• Future releases will be fine and will not lose any data, only these two early alphas are affected
• The minimum supported Android version is now 23 (6.0 / Marshmallow / October 2015)

To opt-in to Alpha development, please visit https://play.google.com/apps/testing/uk.co.otissoft.mybrickset to opt-in to testing releases (same process for Beta releases) and then join the group https://groups.google.com/d/forum/mybrickset-alpha.  The Alpha release 3.0.2 should then be available for you in the Play Store.

If you have any issues or questions, the above Google Group is probably the best place to raise them but I can also be reached through the usual methods: mail, PM, or a post here.

RikTheVeggie

The latest Alpha release 3.0.3 is now available.  If you're already signed up then it should download as usual; if you'd like to sign up then please visit https://play.google.com/apps/testing/uk.co.otissoft.mybrickset and join the group https://groups.google.com/d/forum/mybrickset-alpha - this will then show the Alpha releases in the Play Store.

I'm expecting this release to represent the last of the major changes being made as part of the upgrade from 2.x, and will shortly start looking to roll it out to a wider audience (so long as no issues are reported).

This release includes a lot of performance improvements, especially on first launch or when upgrading.

In a major departure from all other releases, this is the first release in which I no longer ship with images and they will instead download on-demand (and will be cached for future use).  I agonised over this decision for a long time as it means the app no longer supports the ability to view the entire LEGO catalogue whilst offline (technically it still does, just without images). 

Given the constantly growing size of the images however, along with the maintenance overhead and several users complaining about disk space usage, I decided now was the time to bite the bullet.

If everyone feels this was the wrong decision then I am certainly willing to revisit it, but I'm hoping that few users will ever notice the difference except that their downloads from the Play Store are now much smaller.

sid3windr

Would it be possible to have an option to pre-download them ? I don't really care about disk space or bandwidth ;-)

RikTheVeggie

Possibly, but I'd have to give it some thought as to how to do so.  It also reintroduces some of the maintenance overhead as I would have to download and package all new images with each release.

Unless you're asking for an option to download each image one by one from brickset.com and cache them locally?  That could be done but I'd want to check with Huw that it won't leave him with a huge hosting bill!  Would also take quite a while to download images for ~16000 sets!

If talking about downloading a pre-packaged bundle of images (similar to how it works now)..
If I use Google Play to host the expansion files, then they download automatically as the app installs.  So I can't use that for things which are optional.

Possibly I could use a public share on Google Drive, if it supports resuming of downloads (if someone gets 90% of the way through a 200MB+ download then has a connection issue, I wouldn't want to restart from scratch).

Otherwise I'll likely have to look into an FTP host which will likely cost money (my current host has quite a low cap).

Maybe when 3.x is out publicly I'll be able to revisit this question and hopefully come up with a solution.

sid3windr

Aha - I see the simple question does not have a simple answer... Yes, I did envision getting them from Brickset.

Brickset is behind cloudflare so that may not be that big of an issue as many of the images would be cached already, but you're right, it's a LOT of sets.

Maybe there is a way for Huw to provide a zip file somewhere that already contains them... but yeah, it's easier to ask than to think of a workable implementation.

Indeed a shame Google does not allow optional packages!

VorpalRyu

Having issues getting the app to update, took a dozen attempts after I updated my collection on the site with the sets bought during Big W's mid year toy sale (even restarted my phone twice). The app is currently still showing a few sets I bought at Target's end of financial year sale as wanted, even though they are now marked as owned (this is after multiple attempts to sync, including after three restarts of my phone, two days after I first noticed it).

RikTheVeggie

@VorpalRyu
Is this the beta or the latest version from the store?  Either way, it shouldn't be happening.
Can you send me the logs & db please, along with the numbers of a couple of sets that shouldn't be marked as wanted, and I'll take a look.

VorpalRyu

Sent the files via the app, my owned sets list is as follows:
https://brickset.com/sets/ownedby-VorpalRyu

& my wanted list is as follows:
https://brickset.com/sets/wantedby-VorpalRyu

All the currently available Harry Potter sets released this year should be showing as owned, but #75945, #75946 & #75948 all show up as wanted in the app.

VorpalRyu

Also, I'm part of the beta test & currently running the latest version.

RikTheVeggie

FYI I've now started the rollout of 3.0.6 to the beta channel.  It doesn't look like anyone downloaded the app whilst it was in "alpha", so I'll close the alpha channel & group.

I'm currently only targeting (a random) 10% of users who have signed up for beta releases, if I don't hear any complaints or notice any major errors in the online analytics (3.x can now auto-report issues to me) then I'll increase the target %.

If you'd like to get the build without waiting, you should be able to do so from here: https://play.google.com/apps/test/RQdfTsKF8l0/ahAAvZMmiKhDYj_M_XIoVest4DYqdye3_ow18IL5DVjV5iFC0ERqdnUdsqMGGqQk_wF_9BiLhhENYgM_zmVvl4z4HO

RikTheVeggie

Does anyone here get the CAPTCHA when they log in?  I have a user with the latest beta saying that it doesn't work, but the 2.x line does.  This is very hard for me to test (I have to go through a very slow proxy which only works a portion of the time, and then the CAPTCHA times out whilst trying to load) so I'm wondering what other people are seeing.

bennok79

I'm running 3.0.8 and don't get the CAPTCHA when logging in.

mr_leerman

I'm running 3.0.9 and search seems to be completely busted.  I can't get anything to come up when searching by partial name or complete set number.  I can get a portion of the correct results if I search by a partial set number but not the actual set I used the full number for.  How can I help diagnose the issue?

RikTheVeggie

I'm thinking this is due a to bug (re)building the db when the app is first launched after an upgrade - in some circumstances the db will fail to populate correctly (eg, when rotating the phone as it says "initialising").  I'm working on a fix but don't have an ETA as life is getting in the way of my development time!
If you could send me your logs & db please (main screen | menu | send logs) I can confirm this, or you can try a clean install and not touch the phone until the login screen shows - that should allow the db to build correctly.  If you don't want to reinstall, the next release should fix the issue automatically.

mr_leerman

That makes sense. No worries I appreciate the work and, as a dev myself, completely understand balancing against life.

I emailed you my logs in case they help. I'll give the reinstall a shot shortly.  Thanks!

RikTheVeggie

Thanks for the logs.  Not exactly the issue I was thinking of, but similar and will hopefully be resolved by both a clean install and the next update.