• > STAY HOME & STAY SAFE! Dear MobiMembers - We are all aware of the current and serious situation the World is facing right now! We will not repeat what is already said, save to say we hope you will all heed the warnings, stay safe, look after yourselves and others and remain well. As much as we are able, let's all help each other to stay sane in our social distancing/isolation and face this thing down. Much love 'n hugs, Rog, Matt and all your MobiStaff... x x <

Photography for the blind.

RHKing

MobiAddict
Messages
484
Reaction score
1,714
Real Name
Robert H King
Device
iPhone X
I came across this app this morning and it really made me think. It is amazing the many ways that the iPhone/smartphone technology is making many new things possible.
FotoOto by Publicis Pixelpark
https://appsto.re/ca/nOhrfb.i
View attachment 93525
Downloaded this the other day, looks promising. Not convinced at the practicality of how it creates music to match the picture though.
 

FundyBrian

MobiLifer
Mobi Veteran
MobiSupporter
Messages
10,591
Reaction score
31,817
Real Name
Brian Townsend
Device
iPhone 8 Plus
My 365
My MobiTog 365
Downloaded this the other day, looks promising. Not convinced at the practicality of how it creates music to match the picture though.
I expect it is an emerging technology. We have facial recognition. Why not scene recognition? Sunny day, cloudy day. Active/ restful. Any of these clues could generate some sort of mood in music to convey an audio impression. I'm sure any of these would be exciting for someone who could not see at all. To receive any input would be something new to explore.
Imagine the pains we take when we select music to go with pictures in a slide show or video. We have a sense of what fits.
 

FundyBrian

MobiLifer
Mobi Veteran
MobiSupporter
Messages
10,591
Reaction score
31,817
Real Name
Brian Townsend
Device
iPhone 8 Plus
My 365
My MobiTog 365
I've been experimenting with the app. There is a colour-to-audio key that you would need to learn for this to work really well. However, I tried a few sample photos and was surprised at how well it summarized the photo in words. A stem and leaf of a tulip, which I thought somewhat abstract, was recognized as a close-up of a plant. It makes fairly simplified descriptions. A tree with a sunset behind. One of my telephoto shots of a Chickadee on a branch with a black background was described as a small white bird on a branch. Two people standing in the snow. All very good.
It tends to zoom in and crop your photos to vertical.
Then it is quite fascinating to listen to the audio interpretation. It seems to assign sound tones to the size of the coloured areas. Again, you would need to become familiar with the audio palette in order to consistently interpret the images.
There is sort of a brail pattern over the image and it seems to scan the photo where the brail dots are and assign sounds as it scans.
The output of the process is a short video, 13 seconds. I saved one to my camera roll and then converted it to a GIF to send but to do that I will have to leave the app and move over to Safari.
 

FundyBrian

MobiLifer
Mobi Veteran
MobiSupporter
Messages
10,591
Reaction score
31,817
Real Name
Brian Townsend
Device
iPhone 8 Plus
My 365
My MobiTog 365
View attachment 93565
13 second video converted to GIF with audio. Smallest size gif. You can see the dots and see where the sounds are being assigned to the image.
This picture was identified by the app as a close-up of a plant.
Unfortunately it seems the audio in the GIF doesn't play here.
 

RHKing

MobiAddict
Messages
484
Reaction score
1,714
Real Name
Robert H King
Device
iPhone X
I expect it is an emerging technology. We have facial recognition. Why not scene recognition? Sunny day, cloudy day. Active/ restful. Any of these clues could generate some sort of mood in music to convey an audio impression. I'm sure any of these would be exciting for someone who could not see at all. To receive any input would be something new to explore.
Imagine the pains we take when we select music to go with pictures in a slide show or video. We have a sense of what fits.
A very good point well made. Perhaps I was being a bit too cynical?..
 

Ryn S

MobiLurver
MobiSupporter
Messages
1,401
Reaction score
5,555
Real Name
Ryn S
Device
iPhone 7 Plus
My 365
My MobiTog 365
View attachment 93565
13 second video converted to GIF with audio. Smallest size gif. You can see the dots and see where the sounds are being assigned to the image.
This picture was identified by the app as a close-up of a plant.
Unfortunately it seems the audio in the GIF doesn't play here.
Omg, this is sooooo cool!!! Just think of the possibilities! Wow.
 

Lzed

MobiLifer
Mobi Veteran
Messages
8,185
Reaction score
15,213
Real Name
Larry Zasitko
Device
iPad (Generic)
An interesting app...
Creates a 13 second video.
Now, how a blind person is supposed to figure this out is beyond me but basically different colours are a different sound. I have only done a few but still pretty neat and sounds quite different with different pictures. You can record a comment along with it.
When you load an image it tries to figure out what it is and will tell you, sometimes works fine, other times it completely misses. Things like flowers or people it can figure out but sometimes funny results. A Picture of an Anime Girl it thought was a Teddy Bear dressed in a Costume or that a chair in an abandoned building was a picture of Cake with Chocolate. A single image of one of my wife's cousins it said was a man and a women taking a selfie, even though it was only a man's face...

I will keep and play around, some of the "Music" it makes is interesting and I am wondering how this really works for a blend person. Still it may be something to watch for in the future.
 

FundyBrian

MobiLifer
Mobi Veteran
MobiSupporter
Messages
10,591
Reaction score
31,817
Real Name
Brian Townsend
Device
iPhone 8 Plus
My 365
My MobiTog 365
An interesting app...
Creates a 13 second video.
Now, how a blind person is supposed to figure this out is beyond me but basically different colours are a different sound. I have only done a few but still pretty neat and sounds quite different with different pictures. You can record a comment along with it.
When you load an image it tries to figure out what it is and will tell you, sometimes works fine, other times it completely misses. Things like flowers or people it can figure out but sometimes funny results. A Picture of an Anime Girl it thought was a Teddy Bear dressed in a Costume or that a chair in an abandoned building was a picture of Cake with Chocolate. A single image of one of my wife's cousins it said was a man and a women taking a selfie, even though it was only a man's face...

I will keep and play around, some of the "Music" it makes is interesting and I am wondering how this really works for a blend person. Still it may be something to watch for in the future.
I studied the sound pallet for a while and although each colour sounds a bit different it would take a lot of effort to hear the difference with precision. Perhaps if the sounds were more easily differentiated. To me that is the weak spot in all this.
However, overall I think the idea has potential, once the actual "code" to translating visuals to sound is made more recognizable.
 

FundyBrian

MobiLifer
Mobi Veteran
MobiSupporter
Messages
10,591
Reaction score
31,817
Real Name
Brian Townsend
Device
iPhone 8 Plus
My 365
My MobiTog 365
I found another sound/colour app.
Color Sound - Listen to the Colors by Mathieu Bolard
https://appsto.re/ca/iEmVab.i
Unfortunately it is a paid app so you can't just play with it to see how it works. Basically, you hear sounds of a different pitch depending on the colour your lens is pointed at.
 

FundyBrian

MobiLifer
Mobi Veteran
MobiSupporter
Messages
10,591
Reaction score
31,817
Real Name
Brian Townsend
Device
iPhone 8 Plus
My 365
My MobiTog 365
Yes, I read your review of Artful & Cloak. The resolution seems too low and the 1:1 is a deal breaker. Interesting sample pics, though.
 
Top