Jeffrey Martin on mapping Prague’s streets in 1.3 million photos
Jeffrey Martin, a pioneer in the field of 360-degree photography, has been living in Prague for a quarter of a century. Recently his company Mosaic launched a 15-terapixel open-source data set of high resolution street view imagery of the city. How is this incredibly detailed mapping carried out? And who are the many users already accessing the resource, which is free for non-commercial use? I spoke to Martin, who is from the US, in a small park near his office in the Smíchov district.
What brought you to Prague?
“I came to Prague in 1999. I was 23 years old and I wasn’t entirely sure what I wanted to do with my life.
“I got a job teaching English; I took a course before I came to Prague to get certified as a teacher of English as a foreign language.”
Were you already a photographer in those days?
“Yes, I was. I’ve been an avid photographer since I was 10 years old and I’ve always been extremely enthusiastic about photography.
“And I discovered 360-degree photography in around 2003 and I’ve been quite obsessed about it since then.”
Among the things you’ve done was an amazing image in 2011 of the crowd at the FA Cup Final at Wembley in London. You’re a kind of pioneer of 360-degree photography – what drew you to that particular field?
“Well, I got my first digital camera in around 2002 maybe. It had a very primitive panoramic photo function, where you would take a picture and it would superimpose the last picture on top of the next one.
“It immediately struck me as something fundamentally different about digital photography – that it’s kind of fungible and you can merge multiple photos into one photo, but also that a 360-degree photography describes a single place on Earth, and no other.”
Did you start using readily available technology? Or have you been involved in the development of some of the gear that you use now, or have used over the years?
“In the beginning I just used off the shelf cameras and software, although in the very beginning to create a 360-degree photo required using some fairly complicated and hard to use software.
“So I had to search for help on the internet, in various web forums, to figure it out.
“There were a lot of people helping each other figure out how to do it. And a lot of those people that I met back then, in around 2005, are still very good friends today, actually.”
Were you always technically minded? As a kid, for example?
“Not really. I guess I’ve always liked kind of tinkering a little bit.
“But I wasn’t one of those people who would take stuff apart and put it back together. Absolutely not. I wasn’t at that level of proficiency.
“I think I could take stuff apart, but I couldn’t put it back together again.
“I remember once taking apart my dad’s watch and he was extremely angry, because nobody could put it back together again.”
I first noticed your work when you were doing these “360Cities” photos of cities. Was 360Cities, which is an online site, something you developed, or you created?
“Yes, I founded that. It grew out of a project called Prague360.com, which I started in 2005/2006 and it grew into a sort of global platform.
“It’s still going today – 360cities.net – and it’s a publishing and licensing platform for 360-degree photos, kind of a curated platform for high-quality 360-degree photos.
“So yes, I started that.”
How does that work as a business?
“Photographers can publish their images on it for free, and then they can be licensed to advertising companies, editorial companies, educational companies – and that money is shared with the photographers.”
When it comes to 360-degree photos, how has the technology developed over the years? How would one taken today compare to one taken 15 years ago?
“Fifteen years ago you needed to use a digital SLR, with a fish-eye lens. And you would have some mechanical rotator that would rotate the camera around a specific axis, so that there’s no parallax from one image to the next.
“You would then use some software on your PC to stitch them together.
“And these days there are multiple cameras that you can buy off the shelf, and you just push a single button. It has two lenses and it stitches them together on the device, and you can see it on your phone.
“But you can do 360-degree photography today without any technical proficiency whatsoever.”
“Also on the phone itself you can just get an app that will stitch images together, so you can just take a bunch of pictures with your phone and stitch them together, automatically, on the phone itself.
“You can still do it the original way, and it will still give you a level of quality that is much higher than these other, off-the-shelf solutions.
“But you can do it today without any technical proficiency whatsoever.”
Recently your company Mosaic launched a 15 terapixel, free open-source data set of high-resolution street view imagery of Prague. What is the background to this project?
“My company Mosaic, we primarily make 360-degree camera hardware for geospatial applications. So if you think of the Google cars, that have the camera on top of the car, we make cameras like that one.
“We sell them to a lot of different customers all over the world. We ship these cameras to over 40 countries worldwide and typically they are used by companies to do some kind of inspection of the infrastructure: of the roads, of the electrical infrastructure and things like that.
“Our customers do this for municipalities or departments of transportation or other organisations like that.
“Basically, everything that we build – all the roads and everything around, all the electrical lines and poles and everything – we need to obviously maintain; we don’t want it to fall apart, we want to be able to fix it before it falls apart, before it breaks.
“So the old way to do it was you send some people out with clipboards and they check that everything’s OK and make notes.
“And the newer way to do it is you have a driver and you have a car and you create imagery of everything. You can have somebody check that in the office, and this is a lot cheaper, a lot more efficient.
“We ship cameras to over 40 countries worldwide and typically they are used to do some kind of inspection of the infrastructure.”
“This hardware is what we make. So obviously we make this stuff and we have to test it, extensively, which requires us to drive around. And there’s only so many times you can drive around the same block before you go completely crazy.
“So it makes more sense for us to actually map the whole city.
“At a certain point we said to ourselves, We have hundreds of kilometres of data, it’s extremely accurate, it’s extremely high-quality; the GPS accuracy is within a couple of centimetres, the imagery is extremely high resolution – why don’t we do something useful with it?
“We said, Why don’t we make it available for people who want to do something with it?”
And it’s free for non-commercial usage?
“Yes. So we thought, OK, there are researchers who like to have data sets available for research; there’s a lot of interesting, creative things that could be done with this data – we don’t even know what.
“And a lot of people owe a lot to the availability of certain types of open-source software or free open-source data sets. And I think a lot of good can come back.
“Of course part of it is a marketing activity – it gets our name out – but a lot of good can result from allowing people to have access to a huge, interesting, detailed, accurate data set, in this age of AI and computer vision research and everything else.”
So far, who has been making use of this resource?
“As I said, we don’t actually know initially what people might like to do with it, so when we first announced it we got a lot of interesting inquiries.
“Of course there are 3D artists who would like to try to make 3D models of the data, where they don’t have access to the images that they would like in order to try to do it.
“There are AI researchers who would like to figure out how to detect certain objects.
“We have hundreds of kilometres of data, it’s extremely high-quality; the GPS accuracy is within a couple of centimetres.”
“But then we had one person from the neurology department at Harvard studying saccadic eye movements, the way that your eyes sort of twitch when they move: They want to study that, and for that they need lots of 360-degree photos.
“So that’s one example of some completely out-there usage that we didn’t imagine before.
“But so far we’ve had hundreds of people sign up for this, all over the world, and I think mainly for things related to computer vision.”
To go back to the actual production of this street view imagery, how does your car work in terms of how you capture the images, how you save them when you’re on the move and that kind of thing?
“We have the camera, which is about the size of your head maybe and it’s on a sort of a tripod on the roof of the car.
“And there are a few other sensors. There are some very large GPS antennas, there might be a little laser scanner or something like that. And there are a few wires going into the car.
“So yes, it does not look like a typical car and it does sometimes draw a lot of attention.
“People automatically assume that we’re Google and that we’re making Google Street View, which we’re not, but people think we are.
“I’ve heard it kind of varies from country to country but here people kind of love Google Maps and Google Street View and they’re generally very thrilled that they’re going to be there.
“So they wave and they’re quite cheerful to see us.”
No-one’s giving you the finger, or anything like that?
“No, not so much. We’ve heard from people in other countries that people are quite guarded about their privacy and they get more angry people – like in Denmark, for example.
“But here people seem to be very enthusiastic. Especially when there’s a group of school kids – they kind of erupt in enthusiasm and it’s pretty funny.”
How does your product compare to Google Street View?
“It’s very similar. Google typically makes their own hardware for Google Street View, although they do in certain cases source third-party cameras for data collection in countries where they are not legally permitted to do their own first-party data collection, such as India.
“But in general the basic hardware is very similar: There’s a 360-degree camera, there’s a high-end positioning system, involving GPS and motion sensors, and there might be a Lidar, or laser scanner, there.
“So in general it’s very similar.”
How much of Prague have you mapped in this way? And how long did it take you, roughly?
“We’ve mapped about 80 or 90 percent of the city centre, depending on how you define the city centre, but if you imagine the map that tourists get of Prague, so just the immediate centre: Prague 1, a little bit of Letná, Prague 6, 7, Vinohrady and so on.
“So we’ve mapped about 80 percent of that and that took a few weeks, I would say, maybe 60 hours of actual driving time.”
The press material for the project says that you took one million-plus images per metre. How is it even possible to take one million images in one metre?
“I think that might have been not phrased correctly. The total data set is, I think, 1.3 million images.
“There is one set of images every metre. So when the car moved one metre, it made one set of images – and one set of images is a total of 72 megapixels.”
Are there any rights issues, or privacy issues, when you’re doing this kind of work?
“Yes, there are laws that we have to obey. There’s the GDPR laws, so this type of data needs to be anonymised before it is published.
“This is one of the reasons why there was quite a significant delay between when we announced this and when we made the data available.
“Because we had to blur all the faces and license plates from every picture. So from a million pictures we had to run a bunch of analysis on them to identify every face and license plate and to obscure it, in order to comply with GDPR laws.
“So even though we are driving around all the time, capturing all this data and all these pictures, whatever we publish is complying with the relevant GDPR laws.”
Have similar projects been carried out in other European cities or elsewhere in the world?
“By us, not yet. We have a lot of customers who have.
“We have one customer in Denmark who maps the entire nation of Denmark, every two years. They capture something like 98 percent of the publicly accessible roads in Denmark, which is significantly more than what Google does.
“They license this data to many, many different customers, including government customers and commercial customers.”
Do you have any other notable projects on the go?
“We are adding laser scanners, or Lidar, to our products. So in addition to capturing pictures, we’re capturing 3D laser points as well, which makes it much easier to create a 3D model of everything that we’re scanning.
“Although that’s possible to do doing photos only, it’s much easier to do it with lasers. So that’s what we’re testing now.
“And of course we’re always working on new products, making our devices higher resolution and better and easier to use and so on.”
How is AI affecting your industry?
“Generally people are using AI to automate the detection of certain types of things.
“We have a lot of customers who, for example, need to locate every electrical pole in a region. And it might be a big region. It might be the entire state of California, for example, or an entire municipality.
“If you use AI you can train an AI what is an electrical pole, either from the laser point cloud data or from images, and you can automatically find all of the electrical poles in your data set and then geo-locate them – plot them on a map – without any human input.
“In general this works really well for certain kinds of objects, like street signs, or faces and license plates, or electrical poles.
“It is extremely difficult to teach a computer what is and is not a garbage can.”
“It works less well for things like garbage cans. It’s easy for a human to understand what a garbage can, but it is extremely difficult to teach a computer what is and is not a garbage can.
“Because they come in all kinds of shapes and sizes in fact, and there’s no strict shape of what a garbage can is.
“So some things AI can do easily, some things it can’t.”
Do you feel any threat from AI?
“No, I think there’s still a lot of automation that needs to be done in this industry and actually we’re still talking to people who are doing things the old fashioned way, by sending people out with clipboards, and they haven’t even moved into the 21st century.
“So I think AI is helping to automate some of the computer vision tasks that are still quite labour intensive, but these jobs are typically done by teams of people in countries such as India.
“So those companies may be threatened somewhat by AI, but in the end I think they will just end up being more efficient at a certain thing and they’ll be able to do something else instead.”
Personally are you an AI enthusiast? Or are you, like me, deeply scared?
“I think the threat is always about what bad people will do with it, not with the technology itself.
“I think it’s a tool like any other tool and I think there are amazing things that will be done with it.
“I think that we will be able to cure diseases and understand the human brain. I think that we will understand a lot of stuff that we were able to understand before.
“But, there will be bad people who try to do bad things, as always.”
To bring this conversation kind of back around to the beginning, you said you came here in 1999, so you’ve been here 25 years. What do you think Prague has given you in that period?
“Well, Prague has changed a lot – as have I. I was a young adult when I came and now I’m a middle-aged guy with three kids, with three teenagers, who are almost as old as I was when I came.
“Prague is a very kind city. The more I travel and see places, even when I go to places that I really enjoy, I’m always really happy to come to Prague.
“Prague is the right mix of a proper city, and getting things done, and somewhere where it’s possible to enjoy your life.”
“Because it’s a really good place and it’s a great place to live.
“When I go to places like London, for example, I can’t imagine living in such a huge, busy place where everyone looks so stressed out.
“Prague is just sort of the right mix of a proper city, and getting things done, and somewhere where it’s possible to enjoy your life.”
Giving everything that’s happening now in America, does that make you even gladder to not be living in the States any more?
“Yes, I think the thing about the States that gives me the most pause about wanting to live there again is the health care.
“One of the most dystopian things I’ve seen is people crowd-funding healthcare when their lives are threatened; this is so scary to me.
“Living in a place where this is a non-issue gives me a lot of peace of mind.
“And having had three kids, we have had our fair share of visits to the hospital and visits to doctors and it’s always been great.
“So given everything else that’s happening now in the States, yes, this seems like a place of relative peace and calm – and I hope it stays that way.”
