Virtual Reality shakes the world at IDFA

Yesterday i went to an Amazing meetup called:

IDFA Doclab non-fiction storytelling VR meetup

The demo's (a VR interstellar like game, high res painting viewing, realtime 3D stitching, samsung gear VR, watching your hands in VR, combining oculus+leapmotion and more) were very interesting and some of them immersive!

The people were open kind and there was a shared energy of enthousiasm.

The speakers at the evening program blew me away.  It shows that docu makers are putting a very significant effort (driven by personal passion) to create the first building blocks that will start a revolution in VR filmmaking.
Next weekend I am going to be virtually transformed into my naked girlfriend and the otherway arround, and shake my own hand.
Can't wait.

Thank you Daan Kip for organizing this great event.

Some footage of the evening program:
VIDEO:
http://youtu.be/sCQjm40T48k

and:

HackTheBrain

It all started when I unsubscribed from my Hollywood cinema subscription and decided that if I were to spend an evening watching a movie it should enrich my life. One of my best friends Toine started this habbit long ago and has always had amazing stories about documentaries he has watched
To kickstart my new habbit I persuaded some colleagues to join me in watching Tegenlicht documentary at a meetup in Pakhuis de Zwijger

And it had a big impact on me...

It showed me how much one can read from the brain:
Are you sleepy, awake? How concentrated are you? Are you touching something with your fingers and if so what? Are you using any of your muscles? Which ones? What color are you thinking about?

This documentary showed me an even more impressive result:
Two rats, one in Brazil the other in the U.S., where connected with their brain and showed that they could directly influence each other’s brain.
Furthermore, a group of monkeys connected with a brain-net could (from a distance) collaborate and display an increased form of intelligence by solving harder problems that could not be solved individually...

Although these methods required invasive electronics (that go inside the head), braindata can also be recorded in a safe way for living humans. It is derived with large, expensive static scanners in research and hospital labs. Now, however, there is a big trend because they are now being replaced by cheap, mobile, and affordable headsets that are designed to have similar functionality using open platforms :-)
As the Emotive, and Interaxon Muse.

This got me thinking... I am highly conscious of the time/frustration we waste on slow input devices like touch screens, keyboards, mice, remote controls and all that crap. For example, did you know you are now sitting behind a keyboard that was designed to slow you down? This was useful in the early days because it prevented jams in the mechanical typewriter.. Did you also know (as your brain is highly adaptive) in just a few weeks you can type twice as fast (and with much more comfort) with that same keyboard if you change your layout to Dvorak. I did this in first year university and benefit from it everyday.

However maybe we can save you the trouble of changing your keyboard layout: what if you could interact with your computer with a device that directly reads your brainsignals? 
While I was thinking about this in Pakhuis de Zwijger a man called Martijn Arts came on stage and did an amazing pitch about a Hackathon he organized together with The Waag society


I felt destined to participate and couldn't wait to invent useful apps and ideas on top of these brain reading devices. So I decided to form a team and started to recruit the best programmers, neuroscientists and creative artists that I could find. I managed to carry forward my enthusiasm on some colleagues and it all started to roll.

The VicarVision team
In a few weeks 5 colleagues were interested and they brought some friends!
Our technical heart Kasper brought his girlfriend (so cute) who also has an background in Artificial Intelligence. My colleague Sicco came up with a neuroscientist (Arthur) in his friend network. That was just what we need to complete the team.

2 ladies and 5 men strong we started to prepare the hackathon in a few evenings with pizza and beer. It was very energizing to brainstorm the idea's together and very useful to play around and getting to know the software (as you don't want to waste your time on that during the actual hackathon).
  

Team: VicarVision
Sicco van Sas, Arthur Buijink, Tim den Uyl, Mariska Snijdewind, Tjerk Kostelijk
Kristin Rieping, Kasper van Zon

So this weekend we participated with about 60 hackers in the hackathon HackTheBrain located in 'De Waag'. I can write a whole book about it, it was so awesome, but let's show you some projects we did.


BrainTinder
Few months a go my friends showed me an app that has a big success in the dating industry, called Tinder. If you don't know Tinder, it works like this:
It's a kind of modern day 'hot or not'. It presents pictures of other Tinder users based on your location and let's you choose whether you like or dislike them. The ones that likes you back become your matches with whom you can chat to setup a date. 
A lot of user 'hack' the system by liking everything in order to achieve a large number of dates. (Sadly ending up with dates with ones they didn't really like).
We thought that this system was ready for a significant improvement.

After some research we found evidence that we can tap into the brain for the signal that is activated if you care about something and if you love somebody.  The research was performed on babies, but what the hack: we decided to give it a try.

The idea was clear, we wanted to make the first honest Tinder: a Tinder based on brainwaves.

We had three days (and two nights) to setup a complete research pipeline which consists of data collection, inspection and writing a live application. On top of that we had to prepare a professional pitch for the jury.

Luckily we played around with the software in our preparation evenings so we managed to developed a brainrecorder. Right after we finished him we invited participants from the street and in the building to come to our test setup. We automatically showed them 3 minutes of ugly and beautiful people as we recorded their brainwaves and facial expressions.
While we were collecting this data, I took some walks in the building and checked the progress of the other teams. 'De Waag' is really a heaven for a geek, so much expertise and tools available.  

The jury warned us to give a basic output so I decided to pick up that point myself. I found an amazing lasercut machine connected to a computer and Pieter helped me out with cutting out a woodshape. I also found lightpaper so you can emit colored light. Using an unfinished hobby project that I did with Bram were we connected a remote control that can control powersockets with a relay board and a Raspberry Pi. I used it in order to automatically switch on a red and blue light behind the lasercut board if someone does or doesn’t have a match.

 

Back to BrainTinder: as we didn't design the software that reads the brainwave's it was a pain in the ass to analyze the data from the participants. Although an ambitious idea was to use artificial intelligent classifiers on this data, we were getting short on time. And right as we were having some breakthroughs the power went down. And it was not just a fuse... I looked out the window and whole 'nieuwmarkt' had no power. They said this occurred 2 times in the last 10 years.  

We had brought PC's instead of laptops and all our data was stored on these PC's so we couldn't analyze our precious results. The backups were dated 1 day back (and that is a lot in hackathon time space!). BrainTinder was about to fail...

We decided to 'omdenken' the problem and went out the building to walk to a pub to prepare our Pitch (and to suppress our stress with beer :))
As we walked away from the building the power suddenly went back on, but it was too late, the keyman was gone. Luckily we made a deal with 'De Waag' that we could start really early in the morning.

Next morning we were the first in the building and after a good coffee we started in superspeed mode to continue the project. Soon we managed to see a trend in the alpha symmetric brainwaves, especially when we saw ugly people ;-) 
 
Time to start to lay out our rescue plan, a live Tinder without classifiers but with fixed thresholds. Just a half hour before the deadline we managed to finalize it and we had a working BrainTinder.
 
We are pretty proud of the result, a demo video can be seen here.

So about the other projects...


GreenSleep
When you come home and fall a sleep on your couch after a hard day of work it is a waste of energy if your lights/tv are still on.
To tackle this we designed a project that automatically switches your devices off depending on your drowsiness: saves you (and the earth) energy (costs). 
This could also be used to wake-up a driver (e.g. with sounds, vibration, lights) that falls a sleep. We dropped the project as it wouldn't make us win (too easy).


FaceSnap

When you go to a concert nowadays everybody is holding a phone recording footage. Research shows that people who take pictures/video's during an event remember significantly less of their experiences. We are all massively tuning out 'the now'.

 To solve this problem we created a project where a google glass automatically starts recording a video when you experience a certain emotion. At the end of the day you could see all your experiences back annotated by emotion. As we are from VicarVision (makers of FaceReader) we really loved this project.


We used the same alpha assymetric prefontal cortex wave's to detect happy moments and wanted to hack the Google glass. 
 
We were able to push an app to the glass that started recording a video. Also we were able to detected an arousal moment in the brainwaves.
But the communication layer didn't work. As we were totally new with google glass developing it took hours to actually find out that it was the unstable WiFi router that caused the problem. Always work with proper Tech!
The breakthrough is recorded and can be seen at the video's section.

We managed to complete the pipeline and the result was awesome: during testing Sicco walked around the building and only the moments were he was very excited were recorded.
See this video. 


 
By the way: A guy from the audience created this drawing

And the winner is....

Although our pitches went pretty good, we unfortunately didn't win the competition. The lucid dreams project did (see picture below). It was a big disappointment but I really agree that the lucid dreams idea could have a bigger impact on society.

TV
Nevertheless we were going to be on television! So after having great fun with the organization and 'Klokhuis' presenter Bas (who took this photo)....

...we went outside and with so much relieved energy and in 5 minutes we had persuaded a pub owner to broadcast the serie and watched it together with beer and nachos, mjamm.

Awesome Videos
BrainTinder - video demonstration
FaceSnap - audience during Siccos pitch
FaceSnap - making off tvshow
FaceSnap - demo
FaceSnap - pitch
FaceSnap - google glass doorbraak
Wetenschap24 playlist

Tim experiences the Oculus Rift at our neighboring team

Press 
We got some serious press attention, blogs are written about the event and the hackathon was on the dutch television ('Nederland 2').
Bart Meijer (famous of the tv serie Klokhuis) created a TV report. 
The full program can be viewed here:
If you are short in time: Here you can view the summarized first and second day.

http://www.wetenschap24.nl/programmas/de-kennis-van-nu-tv/hack-the-brain/

 

http://www.bright.nl/winnende-brainhack-lucide-dromen-opwekken

http://waag.org/nl/project/hack-brain

Next??
So what's next. Besides sleeeeep, loooots of sleep. 
Well: I have a big interest in continuing the FaceSnap project and also have a new idea which I want to keep secret. Their might come a KickStarter project out of it :)

Cheers!

My Contact details: tjerk@vicarvision.nl