Flashed for advertising

Whatever is technically feasible, people will do it. This is also the case in technology-loving Japan, where four companies from the fields of advertising, data storage, cloud technology and artificial intelligence successfully joined forces last year to drive forward a globally unique project: To develop a system for outdoor advertising based on Deep learning or artificial intelligence. The idea behind the project was to enable advertising campaigns that reach specific target groups at the right time with the right advertising message for them. In this case, the system, which is now ready for use, recognizes vehicle types while driving and presents drivers with the appropriate advertising on large outdoor billboards in real time. The first "smart billboard" is currently being built in Tokyo's Roppongi district and numerous others are planned to be ready for use as early as 2017.

Four companies are involved in the project:

Dentsu is one of the largest advertising and PR companies in the world. Smart Insight offers solutions for data integration and analysis. Cloudian offers intelligent data storage solutions for big data applications and Quanta Cloud Technology offers IT hardware, including high-performance servers and IoT equipment.

Deep learning and big data as the nucleus for future advertising technologies

Many projects based on artificial intelligence and deep learning require and generate a very large amount of data. However, current storage technologies do not allow for satisfactory data management. It is not so much about the actual storage as it is about using the data sensibly and supporting the applications with data in real time. A task that data-intensive projects have not yet been able to solve. Cloudian, a manufacturer of software-defined storage solutions, and Smart Insight, a leading data analysis company, came together at the beginning of last year to develop a practical solution to this problem. Dentsu, one of the world's leading advertising agencies involved in deep learning, was invited to participate in the project as a third party. The data was to be collected by recognizing vehicle types in order to use the findings for outdoor advertising campaigns. Quanta Cloud Technology (QCT) and Intel provided the necessary hardware.

Together, they created a project to test the combination of the respective technologies, which ran from the beginning of February to December 2016 and was intended to make targeted outdoor advertising campaigns possible. Never before had such pioneering technologies as deep learning, software-defined storage and an IoT network been brought together in this form. The adventurous event took place on the city highway in Tokyo. As part of the project, the system successfully recognized several hundred car models and provided corresponding data.

The advertising system of the future?

The test setup for the project was based on an all-weather camera with a telephoto lens. This makes it possible to detect vehicles from a distance of up to 800 meters. The camera was installed on an urban highway in Tokyo and connected to a computer system running the recognition software, which analyzes the images provided and uses them to determine the model of the vehicle and its speed within just one second. In order to identify vehicles, this recognition system had to be fed with an initial set of data. This data came from a separate system that was fed with thousands of images of cars from a used car database and was continuously augmented with images of recognized cars from the recognition system's video.

The new system works like this: Once the recognition system has identified a model from the database as a target for advertising, it triggers an advertisement on an LED outdoor advertising screen, which is then visible for five seconds. This information chain opens up unprecedented opportunities for the advertising industry to target customers more effectively than with traditional outdoor advertising campaigns, which have a lot of wastage.

A single image from the video with the parallel recognition of an Audi A4 Avant and a "delivery van".

Although it would theoretically be possible, in the test phase the system is only programmed to recognize car models and not features of the drivers or license plates, which would of course have serious implications for the privacy of the citizens being filmed. In addition to car models, the system can also automatically measure traffic density and average speed. This information can then also be used for traffic information or traffic optimization and is of course also important for the advertising industry to determine the media value of their messages placed on roads more accurately than with previous methods. The criteria for the success of the project were the accuracy of the detection and its latency as well as the number of successful advertising placements.

Possible scenarios for outdoor advertising based on deep learning technologies

The advertising industry has proven to be extremely creative since its inception. The possibilities of a system with deep learning are practically limitless and offer numerous scenarios for its future application. For example, if the system recognizes a truck towards the end of the day, an advertisement for an energy drink could be displayed, along with the message "It's been a long, hard day, hasn't it?" Or the system recognizes a new model from the luxury segment and displays an advertisement for golf clubs or a luxury watch brand. If an older model is identified, an advertisement for a newer model from a competitor could appear or an advertisement from a used car dealer who buys older models. Car insurers could even display the price of insurance for a particular model in real time. The basis for all these scenarios is deep learning, which can recognize a model regardless of how fast it is going, what color it is, whether it is modified or even damaged.

The project was successfully completed in December 2016 after eleven months of testing. The basis for the deep learning system, which initially only consisted of images of used cars, was expanded over time with more and more images from the video, which constantly improved the recognition accuracy. A technical problem also emerged during the test: The identification rate of cars has so far been significantly lower at night than during the day.

What role does software-defined storage play in deep learning projects?
The passive storage of data for later analysis is not really difficult in principle. However, significantly more effort is required to be able to benefit from this data. This becomes even more difficult if it is to be done in real time. The aim is therefore to generate smart data from simple data.

The technical challenges for the storage system are enormous. Continuous filming produces a huge amount of data that requires efficient storage and must guarantee easy access to the data for the recognition software. The data must also be processed in an intelligent way so that the applications can also use it. In other words, it must be usable as smart data. In order to realize deep learning for this system, Cloudian developed the storage to provide the master data consisting of a very large amount of images of cars for the deep learning environment. The master data is constantly being expanded, which further improves the recognition accuracy of deep learning. The object store is based on Cloudian HyperStore software and standard servers and at the start housed more than 23,000 images of more than 230 models. Over the course of the project, the number of recognized models grew to over 300, with a correspondingly high number of images of these models. In addition, the object memory stores the complete video and all statistical data such as the number of cars, driving speed, models per hour and other potentially important data for advertisers. In order to improve the accuracy and speed of recognition through trial and error, it is necessary to constantly adjust parameters such as image types and image quality. To achieve this, a great deal of data is generated and only intelligent data storage systems based on modern object storage can manage such data volumes efficiently.

Deep learning for advertising. A model for the future?

After the project was successfully completed, the project participants prepared the technology for practical use in various industries and the first smart billboard in Japan will certainly not be the only one. Artificial intelligence and deep learning have been constantly improving in recent years and systems based on these technologies not only recognize cars, but also people. Although this has already sparked an ethical debate and raises questions about people's privacy, in the future it will be possible for advertisers to recognize customers in shopping malls and at least assign them to different target groups. Applications that perform tasks for people have already found their way into our lives. Siri, Cortana and Alexa are the result of deep learning projects for speech recognition. Self-driving cars rely heavily on object recognition to navigate their way through traffic. Doctors need the help of machines to recognize cancer cells on thousands of computer-generated images after a CT scan.

Regardless of the application, all of these technologies require huge amounts of data that must somehow be stored and turn the dumb bits and bytes into "smart data". Only very advanced storage solutions can enable deep learning and help to make life more convenient for people - personalized advertising is just the first step. (pd/hae)