It’s been a while since I’ve talked about our “reality capture” (or 3D reconstruction) technologies, but I still like to play around with them, when I get the chance. My previous projects have all been with the PC version of 123D Catch (previously known as the Photo Scene Editor for Project Photofly), but I’ve just spent a few hours messing around with the new web and iPad versions of this tool (Shaan has already spent some time talking about these new versions over on his blog, in case).
Thinking about it, these two new versions have been made possible by exactly the principle we’ve been looking at in the posts over the last few weeks: the technology that currently powers Photofly was originally desktop-based, but was migrated to work in the cloud. The Photo Scene Editor was the first client to make use of this service, before it was even properly exposed via a RESTful web-service API. These new clients – one browser-based, one for camera-enabled iPad devices – were made possible by the exposure and formalisation of this web-service (something that I’ll talk more about in future posts, no doubt).
But anyway – that’s hopefully a helpful example of how the cloud can help scale and broaden the use of technology, but is a little besides the point of this post. This post is really just about how cool this technology is. :-)
To step through the differences between the versions of 123D Catch, I decided to capture a scene that I’ve been meaning to, for some time:
The male members of my immediate family (i.e. my sons and I – and probably only because my daughter is still a little young) are huge LEGO fans. The above creation – with its associated, epic storyline – was developed over a number of weeks (maybe even months) by my 7-year-old. Rather than let it get recycled (the good and bad thing about LEGO), I decided to preserve it digitally for posterity (and eventual re-creation in case of accident, like that would ever happen ;-).
On the subject of LEGO – although this is more related to 3D printing than it is to 3D capture – I just found out about the ingenious Free Universal Construction Kit, which allows you to 3D print connecting pieces to combine LEGO with other building block systems – several of which I’ve never heard of: Duplo, Fischertechnik, Gears! Gears! Gears!, K’Nex, Krinkles (Bristle Blocks), Lincoln Logs, Tinkertoys, Zome, and Zoob. I love the idea, although a) apparently today’s domestic 3D printing technology – at a precision of around 100 microns – may not be enough for some block systems (LEGO is manufactured at a precision of 10 microns, apparently) and b) we really don’t have any sets of the systems the kit allows you to connect with. Which clearly defeats the object for my household, at least.
OK, back to today’s topic…
I started with the iPad version of 123D Catch – which it seems is currently only available in North America (one of my iTunes accounts is US-based, which means I could download and try the app). As you might expect, the iPad version provides a very streamlined experience: it allows you to take photos directly using the iPad’s camera (you’ll need at least an iPad 2, of course, but “the new iPad” will apparently give better results), and after a little bit of meta-data entry (name, category, tags, description) it goes ahead and uploads the scene for processing. Once the scene has been created, you can view in directly on the iPad – and it has a cute “gyro” capability to automatically adjust the view as you move/tilt the device – or you can email the scene to open it up in the desktop version (which is particularly helpful if you want to edit the scene, stitching in images, etc.).
I found that the quality of the camera on my iPad 2 wasn’t really good enough to capture models adequately, especially in low-light conditions. And as this version has a limit of 40 images, you’re less likely to capture the level of detail that you might want from a complex model.
Here’s an image captured from the iPad 2, to give you an idea of the quality:
I ended up switching across to the web version (which also has some pretty cool, browser-based visualization that must use something like WebGL – something we’ll take a look at next week, incidentally) to create a scene based on images I took with a more capable digital camera. Which was all well and good, until I realised that I’d quickly taken 256 MB (purely a coincidence – my inner geek wasn’t especially aiming for that number :-) of photos, while the web version limits uploads to 200 MB.
Rather than resize the images (which I vaguely remember might affect the EXIF data that helps Photofly reconstruct the 3D scene – although I’m probably confusing that with rotation or cropping) or reduce the set of images selectively, I went ahead and uploaded what I could:
Something to note: I created a few different photo scenes with different subsets of the images, to see what happened. While the desktop version can tell which images have already been uploaded to the web-service, it seems the web version does not (which means more time is spent for repeated attempts, and may well be part of the reason for the 200 MB upload limit).
Once the upload was completed, I received a message stating the scene was being processed:
Once the process has completed, you’ll see the status in the capture’s description update to state “the capture is ready”:
For some reason – while the capture has worked for me with other sets of images – these particular images (or the quantity of them, perhaps) have not resulted in a working set (and that’s even true for the set that is flagged as ready). Given the lateness of the hour, I’m going to have to leave it there, at this stage, and take another look at some point in the future.
So with that, on to my results with the desktop version…
I uploaded the full set of photos, and waited for the processing to complete:
Once completed, I had my scene, which was pretty close to what I was looking for:
I suspect that with images taken with better lighting – and some editing of the scene, to clean things up – the results would be even better. And that may also be true of the web-based process. But this will already serve as a pretty good reminder of a child’s creativity, which is ultimately what I was looking for.
Well, that’s it for my whirlwind look at the three available versions of 123D Catch. If you’re interested, go ahead and check out the version that suits you best – and be sure to let us know where you’d like us to go with this technology.
