This Swiss legal-tech startup converts flight delays & cancellations into cash. Interview with Edoardo Köppel,…
Dr. Nikolay Kobyshev
Interview with the founder of a startup that revolutionized the airline industry through computer vision
Artificial intelligence, future technologies, computer vision – these futuristic words bring enormous popularity to sci-fi films. With interest and excitement, we watch ‘Black Mirror’, imagining how computers will change our future and what role will be given to us in it. Meanwhile, somebody is creating this future right now and changing entire industries literally from the neighboring house.
Nikolay Kobyshev is a Doctor of Computer Science, ETH Zurich alumni, founder of the startup, Assaia International AG, and board member of Spectando. A few years ago, he moved from Russia to Switzerland to study at a leading university, and today he runs a global innovation business. In this interview, Nikolay talks about why algorithms are more reliable than people, how education impedes success, why airlines save on olives, and whether or not we should be afraid of technological progress.
What influenced your choice of profession?
When I was 15, I dreamt of an interesting life – one that would allow me not to regret my choice after a couple of decades. I always liked computer technology. I thought, and still believe, that this is not only a very promising profession, but also a field that is really beneficial for society and that objectively changes our world. Besides, I liked the mathematics and logic of its application in the computer sphere: you enter the formula, give the command, and the computer performs it exactly. With this motivation, I entered the Faculty of Technical Cybernetics of St. Petersburg Polytechnic University. At the same time, I also studied for a bachelor’s degree in economics.
Why did you move to Switzerland?
When I completed my bachelor’s program, I didn’t really know what to do next. My grades were better than those of others, but this did not affect my career prospects in any way. Employers value the experience of a person much more than his or her academic performance. The only way to implement the academic advantage was to continue studying, trying to enter a top university. That’s why I applied to various European universities. It all worked out for the best: I was accepted into the ETH Master’s program and even received the Excellence Scholarship and Opportunity Award, which is a kind of grant.
I’m glad I got a good education. However, I think that education by itself is useless in life.
Looking back, can you say that ETH Zurich was the right choice?
I suppose so. I say ‘Suppose’ because I don’t have anything to compare it with. I’m glad I got a good education. However, I think that education by itself is useless in life. Admittedly, it develops the ability to work and gives a deep understanding of your subject, but this academic knowledge may not be so important. There are many examples of well-known people whose lack of education has not prevented them from becoming extremely successful, for example, Zuckerberg, Gates, and Jobs. Besides, the academic path is sometimes even a hindrance. Of course, you develop, you learn to think logically, but you waste valuable time studying.
Realizing that a long education is not essential for success, you nevertheless went even further and enrolled in a PhD program. It’s a big step, what made you do it?
Actually, a PhD seems to me less categorical than working straight after graduation, because it does not deprive you of choice. On the contrary, it gives you a great opportunity to do something interesting and useful without closing your career doors or burning bridges. For instance, after acquiring my PhD, I could have worked as a programmer, or gone into business, or been involved in scientific research – that is, the horizons were expanding.
In my case, it was like this: after obtaining a master’s degree, I realized that I had a good enough situation – I had already lived in Switzerland, successfully graduated and knew for sure that I could find a job in my field. But just like many years ago, I wanted my career to be something non-trivial. In addition, I had a childhood dream to do science; to gain deep knowledge in some area. I wanted to satisfy this curiosity, and I am glad that I did it to the fullest.
What was your specialty during the PhD?
Already during my master’s degree I studied computer vision. Initially, I chose this area simply because it seemed to me to be both interesting and promising. Eventually, I really liked the specialization, so I continued to work on it during my PhD. But a year after the beginning of my doctoral studies, the absolute boom of computer vision happened: there appeared deep neural networks, which turned the industry upside down and allowed me to think seriously about the implementation of computer vision in broad areas.
In simple terms, what is computer vision?
It is computer decoding of visual information, so to speak. There are tasks that have been solved with its help for a long time, the most common of which is the barcode; the symbols of which the computer understands very well. Then there was great success with the recognition of indexes on the envelope and car license plates. Then there appeared good facial recognition methods, which made it possible to add the face autofocus function to cameras. But the tasks that sounded super trivial to a person, for example, distinguishing a picture of a flower from a picture of a dog, were impossible for a computer until about 2012. It was then that deep neural networks brought computer vision to a new technological level.
How do deep neural networks work?
Computer neural networks work in the same way as the human brain, as much as it is studied today. The network consists of a huge number of interconnected artificial neurons, which are constantly and independently trained on which electrical signals they are stimulated by and which not. For example, one neuron evaluates the brightness of a small group of pixels in an image and triggers if it becomes higher than a certain point or doesn’t react when the values are lower than this point. Within the network, there are millions of neurons that are interconnected and activated depending on the activity of the others nearby. There is the last neuron on the output that is triggered if there is a flower in the picture, and it is not triggered if there is a dog in it.
Initially, neural networks are programmed only for which neuron speaks to which one. But when it will or will not listen to the previous neuron, is part of the training. Thus, neurons independently try to find the most effective way to solve the problem. During training, large amounts of data are loaded into the neural network for analysis. The more input data the network receives, the better it will work and the more accurate results it can achieve. This is what artificial intelligence is all about with the example of neural networks.
If there is incontrovertible evidence that a machine works better than a human being, I would definitely trust the machine more.
With the innocuous sorting of the pictures everything is clear. But do you think modern systems are sophisticated enough so that people can rely on them to make really serious decisions?
Algorithmic errors usually become resonant, for example, a network that recognized a woman as a gorilla, or a network that identified innocent people as criminals. However, it is very important to understand that the network does not want to offend anyone, it operates on the basis of the data that has been uploaded into it. If the input data was insufficient or incorrect, the network will only repeat it and increase human error. This is very similar to kids raised by bad parents.
I believe that trusting algorithms is a matter of gaining knowledge. If there is incontrovertible evidence that a machine works better than a human being, I would definitely trust the machine more. That is, if the system recognizes tumors from a scan better than a human, I would insist that my scans were examined by the machine, simply because this choice will give me a better chance of survival.
What do you think of the idea of incorporating computer control into everyday life, such as the social rating system in China, which scares many people?
On the one hand, I see certain positive aspects of this approach, for example, if the system would help to track down and catch criminals. But I would only want to live in such a society if I trust the author of these technologies 100 percent. I think, however, this will never be the case in any society.
If we talk about technical innovations in general, I am, of course, always on the side of progress. I understand that often the unknown scares people, but I think that the point of no-return has passed. Technology is rapidly changing the world, making it easier and more accessible. Today you get to know each other through Tinder, communicate through Facebook, go home via Uber, and call anywhere in the world through WhatsApp. Of course, there are both pros and cons to this, but the technology is still too useful, so there’s no point in being afraid of it or restraining its development.
You’ve been personally involved in the development of technology for many years. What projects have you been working on?
Yes, I have always tried to do something in parallel with my studies. At first, we made Hackathon HackZurich, a competition for programmers, where they have 48 hours to come up with an idea and implement it in a prototype. This format contributes greatly to the creative process, which is often blocked in everyday life because, usually, if you want to create a good product, you need to make it qualitatively. Development takes a lot of time, and you still don’t know whether the product itself is worth it. In Hackathon mode, you say in advance: I will do it somehow, to make the prototype work in some way. This helps to generate interesting ideas.
Also, during my studies, I created my first independent project, the Spectando company, which specializes in virtual reality for real estate visualization. The idea is that when you’re looking for an apartment, it takes a lot of time to visit the properties. We provide an opportunity to view virtual analogues of apartments to narrow the focus of the choice and immediately determine where you want to go, and what options you are not interested in. Today, Spectando is a stable company with a number of clients; I’m still on the board of directors.
Your main business today is ASSAIA. When and how did you come up with the idea for it?
In October 2017, I finished my PhD and I knew that I wanted to work with computer vision in the future. This desire was shared with me by my old friend, Dmitry, with whom I have already done several joint projects, along with our business partner, Max. We believe in computer vision and that it can radically change certain industries. That’s why we decided to create together a company that offers this technology.
At the time, we did not know which industry to focus on, but we targeted the B2B market from the very beginning. To determine the direction to go in, we offered different products and simply researched the market as a whole, trying to find out where we could be useful.
We are talking about an industry where American Airlines saved $40,000 a year by just reducing the number of olives by one for first-class passengers’ lunches.
In other words, instead of developing a product and starting to offer it to businesses, you did the opposite: you asked companies what they needed and based on this you developed a solution?
Yes, and I will do exactly the same for all future businesses, because this is a more sustainable model. To compare, we built Spectando around an idea. In retrospect, I understand that throughout the development of the product, we tried to convince ourselves that the market needed it. Now I think it’s more appropriate to find what the market needs and create a customized product on that basis.
How did you analyze the market to find your niche?
We acted in a standard way: we organized workshops with customers and just phoned the companies, trying to understand what solutions were needed; then we came up with some products and watched how well they sold.
All this eventually led us to the airline industry. There are many factors that have shown us that this market has a great need for optimization. Firstly, this is measured in subjective things: how quickly do you receive a response to a letter, how much fire do people have in their eyes? But the objective signals, of course, also confirm the existence of a problem. After all, we are talking about an industry where American Airlines saved $40,000 a year by just reducing the number of olives by one for first-class passengers’ lunches. It doesn’t take much analysis to realize that maximum cost savings on processes are essential for survival in this market.
Optimization of which processes do you work on?
Currently, we focus on turnaround: all that happens to a plane after it lands at the airport. Now it’s a real black box: almost nobody knows exactly what happens to the plane and how to control it. However, the understanding of this is critical, because any inconsistency can lead to a delay in departure, which causes enormous losses.
The second aspect is security problems. Incidents of all kinds often occur on the airfield, from situations where it is forgotten to load luggage on board, to physical collisions between aircraft. Finally, there are difficulties with airport capacity. The number of flights is increasing and is expected to double within the next 20 years. But the capacity of airports remains the same, because physically growing an airport is a very difficult task. Therefore, the only way to increase the capacity of an airport is to increase the efficiency of its internal processes.
This is where I see the beauty of working with data: you can change the current industry not with the help of new mechanisms, but simply by using the data.
What product does ASSAIA offer, and what problems does it help to solve?
First of all, we enable customers to get a very clear picture of what happens to an aircraft during turnaround without adding complex sensors, but simply by using an installed video camera. This system determines when a plane arrives, when passengers start to leave, when cleaning, refueling, electricity connection and a hundred other things begin. This is no exaggeration: the list of parameters that we can control includes 110 indicators. Such a monitoring system allows for more accurate decisions when coordinating.
Secondly, we offer a decision support system that advises flight operators about which aircraft to pay more attention to and which aircraft are at risk of being delayed. We have three objectives: to increase efficiency, safety and the environmental friendliness of aircraft turnover at the airport.
How exactly does artificial intelligence work in your system?
The solution is 100 percent tied to artificial intelligence, which works on two levels here. On the one hand, our computer vision system, just like with the example of a dog and a flower, looks at the picture of an airplane and determines what is happening to it at any given moment. On the other hand, we have a system that understands how the airport works in general and can predict what will happen to it within half an hour, an hour and so on. Its accuracy is already higher than the human method and it is optimized every day.
Who are your clients?
We have three key types of customers: airports that want to improve their capacity and efficiency, airlines that are interested in the same thing, and handlers who deal directly with operations (luggage, passengers, etc.). They all benefit from greater transparency and predictability.
Our system is versatile, suitable for all airports and companies, and the technical requirements for connection are minimal: you just need to install a camera that will see the plane.
Do you have any competitors?
As far as we know, we are the only company in the world that deals with artificial intelligence systems exclusively for airports. This is a niche product, and we believe that such a choice – to make a quality product for specific customers – is optimal.
What are your plans for the next five to ten years?
We are already working at a global level, and our growth plan is to become world leaders. I think we have a good chance, because we work in a specific niche and perform our function well. Therefore, I can be quite optimistic and expect that in the future our system will be installed by default in all airports.
In addition, we are striving to understand even better how an airport functions in order to connect all the technical systems into one. This will allow us not only to get crystal-clear information about all the processes, but also to predict the development of events when certain parameters change. The more data a system can analyze, the more users benefit from it so they can optimize their operations. This is one of the priority directions of our development.
How do you see the future of the aviation industry in general? What changes can we expect?
Today, there are certain environmental problems in the aviation industry that will surely be addressed in the coming decades. In addition, the number of aircraft will grow rapidly, which will require even greater optimization of all processes. In fact, I am very happy that the industry is set to grow. I believe networks that unite people are very important for the development of society. The main network is, of course, the Internet, but the physical transportation of people and objects also plays a huge role.
Could it happen that planes will become outdated and be replaced by a fundamentally new type of transport?
I don’t think that in the next 10-20 years planes will become outdated. There is no reason for radically new technologies to emerge. The aviation industry is a rather conservative market, and this can be seen in its history: 20 years ago, for example, aircraft did not differ much from today’s aircraft. By the way, this is where I see the beauty of working with data: you can change the current industry not with the help of new mechanisms, but simply by using the data.
How will international data exchange transform – will it become more open?
The advent of the Internet has already made data exchange unrestricted for people worldwide. More difficult, of course, is sensitive data – no one wants to share it, but everyone wants to get knowledge synthesized from it. In this sense, machine learning is a good compromise, because nobody is interested in raw data itself; what can be done with it is interesting. If there is a machine that analyses the data and displays knowledge that is useful for everyone, then it is great. And it contributes hugely to the development of mankind.
Well, let mankind develop! Thank you for your interview and all the success in your projects!