The Journey of Developing Cognitive Functions for Dreams4cars through Simulation
01/06/2018
Henrik Svensson is a cognitive scientist based at the University of Skovde in Sweden with a PhD background in cognitive studies. Henrik brings his speciality in cognitive science to the Dreams4cars (D4C) project and with that his work in building computational, not neuroscientific, models of how the brain and cognition works as well as developing better cognition theories.
Cognition is about simulation
He has studied simulation theory extensively, which basically states that higher level cognition consists of simulated chains of actions and perceptions. We move around the world and see things, but we replay them later, off line, in the brain. We can do it very explicitly, when for example we have mental imagery, but it can also work implicitly or unconsciously when we simulate things very rapidly and predict what will happen if we do certain things.
According to Henrik, simulation is something that the brain does in many different circumstances and many different ways, but we have this basic kind of simulation mechanism that underlies much of the cognitive phenomena that we see. It’s a combination of mathematics and biology. It’s based on neuroscientific evidence but also psychological experiments.
The close proximity of simulation to reality
There have been many studies that show how simulation in the brain produces very similar results to reality itself. For example, you can compare being on a treadmill and imagining it. Not only are there correspondences between the time it takes to actually imagine running, but respiratory and heart rate increase also. So in simulation the entire pulmonary/respiratory system is being reactivated. Interestingly enough, the brain simulates and acts reality out in the same parts of the brain. This is a tip of the hat to Hesslow’s theory, foundational to the D4C project and personnel involved.
Hesslow modernised the 5 or 6 theories regarding brain simulation, which start from the proposal that thinking is simulated interaction with the environment. His primary interest is in the biological side, whereas Henrik and D4C are also interested in the computational aspect too.
Reproducing the brain’s cognitive functions
The ultimate goal in cognitive science is to create computational systems that can reproduce the brain’s cognitive functions. Henrik began this journey during his PhD studies, by building a simple environment and simple robot. D4C is clearly a few steps on from that, with the more realistic domain of autonomous driving presenting a worthy challenge.
In fact as far as Henrik is concerned, autonomous driving is a very good test bed for cognitive science in areas such as perception, attention, decision making and problem solving. D4C will show just how much cognitive ability machines have really developed because there are so many cognitive aspects needed to drive the car and communicate with other cars and road users, as well as their implementation.
Dreams4cars will push cognitive science as far as it can go
How far D4C is able to develop a cognitive car brain is a key aspect of the project. Artificial cognition is no doubt increasing in power and most recently through the increased focus on deep learning, which means neural networks, which are able to learn more efficiently than they have been able to before. When it comes to achieving the power necessary to adequately perform all the functions of driving a car the stark truth, according to Henrik, is that there is still someway to go. One of the issues is the ability to plan and control behaviour, both in the short and long term. The D4C project is able to use systems where the agent can set goals and try out different functions itself, enabling it to begin planning route responses.
The biggest barrier is that AI systems in general focus on one cognitive ability, whereas humans can focus on many and perform lots of different tasks. Humans can focus elsewhere and then refocus on the previous task. AI systems struggle to do that.
Developing a virtual dream world for extensive simulation
It’s time for a quick recap of the project. The dreams4cars project is named after its dreamlike simulations that take place in a virtual world. Since simulation is the key to developing cognitive ability, placing the driving agent in a world where it can try out as many scenarios as it likes without any risk or damage whatsoever makes the virtual world the ideal candidate for the place where these scenarios can take place. What have dreams got to do with it then? Human dreams are forms of simulations that are out-of-the-box, transcendent, extraordinary, and the D4C project is all about thoroughly training the driving agent by exposing it to such kinds of out-of-the-box scenarios.
Henrik is responsible for creating the dream generation mechanism, which creates good learning scenarios for the agent to learn from. This means developing mechanisms that create dream environments, which has now become the Open DS (Open source Drive Simulator) system. This all happens through computer programming. Henrik is working closely with DFKI, one of the project partners, who actually create Open DS.
D4C will actually produce videos of its dream scenarios
Deep neural networks are key in the dreaming mechanism and they are now more powerful than ever before. It means that they can now generate predictions of what will happen as real images in video format, although in their current state are not clear, looking something like hallucinations, or even human dreams, which are random and unorderly. However there is recognisable form and the development team can see how the system recreates or recombines different aspects of what it’s seen before. It is Henrik’s colleagues at Trento University who are working on this exciting aspect of the project.
Self-generating dream scenarios means superior testing
It’s inherently difficult to engineer good learning situations, so having this automatic kind of random process where lots of different scenarios are generated and become novel situations is advantageous, not only because it isn’t easily creatable in the real environment, but because a very broad range of scenarios are created. The more scenarios, the safer the agent will operate on a real road.
Simulation is standard in autonomous vehicle development, but Dreams4cars’ great advantage is the self-generating aspect of its simulation scenarios, whereas in all other cases each simulation has been hardcoded or programmed into it.
Furthermore, from a commercial aspect, if this dreaming really works and corresponds well with real road tests it has the potential to reduce the technical tests needed to prove that the system is running safely enough to drive passengers. This is one of the main aims of the project, to enable economic and sustainable, yet thorough testing.
The dreaming process step-by-step
The overall process is that they have a dream generating mechanism which creates an xml file, specifying how the world layout is, road geometry, number of bends, how many straight sections, etc. The system takes input in the form of what is called a scenario message, which is basically data from lots of different sensors like distance sensors. Objects such as other cars, bikes and pedestrians you’d find in the real world are placed digitally in this virtual world and the agent is given instructions such as staying in lane or overtaking. The scenarios then enable the car to perceive what it would do in each situation – this is what it in essence “dreams” about. This is a combination of computer programming, neural networks, models, and algorithms.
They’re using a mechanism whereby they try to recombine different aspects of scenarios from say driving to work on local roads and motorways to create new scenarios and come up with interesting situations that are hard to create in reality.
They start with their agent in the virtual world and record what happens. Mechanisms are developed that automatically analyse the performance of the car, whether it crashes or behaves well, or has difficulty staying in lane. Each scenario is given two ratings: its difficulty as an environment and how much the car can learn from it. In easy scenarios the agent is simply rated on how good or bad it performed. The agent itself will then try to generate a more difficult world because it’s been able to handle the previous one. The agent is essentially generating the new dream. In a simple case this would mean that it would add more bends to the roads, and then more cars around it. If it manages to interact with other cars well more aspects will be added.
One year in to the project and the virtual dream world is ready for business
Open DS, the driving simulator, is operational at a basic level. Although not everything is implemented or optimised, roads can be created with the dream generating mechanisms. Data can be analysed from actual driving within Open DS and more complex models created. The agent is already able to adapt and dream for itself, but with assistance. A goal in the near future is to develop a scoring system, which will enable automatic generation of more complex roads. The current phase is all about ironing out glitches and fine-tuning.
Back and forth between dream and wake state
In the project open DS testing is as significant as real road testing. In fact one leads to another and is dependent on the other for direct progress, because each needs to validate the other. There’s no point having a perfect simulation if the agent can’t do on the road what it does in simulation. So to align the two driving states the plan is for there to be a cycle of actual road testing (also called on-line or awake) and then simulation (dream or off-line state) and back to road testing and so on.
Work still needs to be done on corresponding the simulated world and the real world, because there are so many situations to deal with. The simulated world has less noise, for example being more synthetic. They have to test and fine-tune the agent and systems so that the agent behaves in the same way on the road as the simulator. Parity is the key. The logic of the project is that the majority of the testing is done in Open DS and a limited level of real road testing. Furthermore, as mentioned in other articles related to the D4C project, a further level of validation will take place through CRF, Fiat’s research facility and a project partner, on their Car Maker simulator. This will act as an independent verification process to further validate that the agent is being trained thoroughly.
Integrating and synchronising all the components together is key
A significant focus of the project a year on in has been on actually integrating everything. There are many different parts of the project to fit together and inter-communicate: the agent, the Open DS system, the dream generating system and the action-selection hierarchy. These components are now communicating within the Open DS virtual world as the basic system connector and testing will indicate further how these components can interact better, what makes a good testing scenario and what mechanisms are needed to automatically create the virtual environment?
D4C is a significant link in the chain to achieve full vehicle autonomy
The D4C project has built on work done in previous projects, such as “Adaptive,” a large flagship project funded by the European Commission. Future projects may continue the work that D4C will achieve in its three-year life. From a cognitive development point of view Henrik expects the D4C agent to fully generate new dreams – its own learning scenarios and to have an increased ability to plot its own course. It is also very likely to see the agent operate in the two test cars, so that they can see how closely the simulation and real world match.
It is unlikely that D4C will produce a perfectly ready, fully autonomous (level 5) vehicle - we should remind ourselves that no one has even come close to doing so this decade - but, innovative systems will be available that will allow OEMs to significantly bring that target closer.
Working across borders has inspired the D4C team
For Henrik, as a cognitive scientist, the experience so far has been a very interesting test bed for cross border co-operation and co-ordination. He has found the diversity within the team of neuroscientists, engineers, driving specialists, mathematicians and cognitive scientists inspiring, stimulating and educational. Autonomous driving is one of the world’s greatest technological challenges and to finally achieve it will require the kind of environment and knowhow Dreams4cars has bred and continues to breed.