Webinar Recording – AI, Robots and the Sense of Smell

We started with closed spaces because it is easier to control (no wind, just HVAC). These methods extrapolate to outdoor spaces, but wind models are needed to inform plume tracking better.

All sensors are commercial sensors purchased from third parties. Functionalization has been done in-house.

The rotor wash of the drone does interfere with the flow. It is important to extend the sensors away from the rotors and model changes in ground effect. Javier Burgess has some great work here.

I am currently monitoring part-per-million and part-per-billion concentrations. I am working harder to track part-per-trillion concentrations and understand sparse odour/plume models better.

Both sensor types drift. Metal oxide sensors can be reset through heating, and Nik Dennler has some great work here. Electrochemical sensors are, in my experience, usually easier to replace after several uses to minimize drift.

To have performance of a human-like nose, I think we will need hundreds. But we should strive for superhuman olfactory performance, like that of a hound, in which case we will need thousands.

Yes, while some adjustments are needed, in my experience, much of the sparse modeling work done in computer vision extrapolates well to olfaction.

Yes, this actually makes things a bit easier because it is difficult to get repeatable absolute concentration measurements. Measuring gradients is easier and more repeatable, in my experience.

Yes, the antennae are used for VOC tracking. I typically start experiments by tracking ethanol as this allows me to maintain consistency with the state of the art from Burgues, Duisterhof, Shigaki, et al.

Yes, the Sensit Smart supports single use electrochemical sensors.

Yes, a digital twin would help, and we do many simulations to assess how the real robot will perform. However, there is a big gap between simulation and reality in robotics, so the simulation is always off. So we must incorporate good uncertainty quantifiers into our models to ensure we model the real world correctly.

We have not tried many pre-concentration protocols – this would be giving some focus. I agree that the sampling technique does make a big impact. We use several electrochemical techniques within our analyses.

One sensors fits in the Sensit Smart.

Yes, there is an electrolyte on every electrochemical sensor targeting a specific compound. The drone/robot continuously samples as it moves about the world and either (a) maps the concentrations of the target compound, or (b) scouts out the source of the target compound.

Yes, we have a bit of work here. You would want to detect the gunshot residue by proxy, or a major chemical component of gunshot residue. One big opportunity here is being able to understand how long the gunshot residue has been in the air by modeling the chemical breakdown of the residue over time and understanding how this relates to the proxy compound concentration.

This is where modern ML comes in handy. We need to deconvolute all signals and attenuate the ones we are interested in. Filtering methods help de-noise the signals and ML models can help separate them. You don’t need thousand-layer transformers to do this. Even small multilayer perceptron models can do very well here.

Yes, vision is the grounding modality for the model that is currently deployed for navigation. For chatbots, language is better as the grounding modality because humans communicate scents in aromas or lingual descriptors – language acts as a bridge between vision and olfaction here.

The human olfaction system is still not 100% understood, so to develop a full nose prosthetic is still a huge opportunity in science. There are the shape theory and vibration theories of olfaction that you can look into to get an idea of the different ideas about how the human nose is suspected to work. I think that a true human nose prothesis will be a combination of electrochemical, optical, MOX, and optical sensors.

Yes, we have some work here with Shalini Prasad’s group. You don’t necessarily want to detect the pathogen itself, but a proxy of the pathogen, or a chemical that the body emits when exposed to the pathogen. Often this is easier than directly detecting the pathogen and gives you more options for sensing.

Yes, I think this would require a nose prosthetic or a brain-computer interface. The digital nose could interpret the aromas for you, but to send them directly to your olfactory bulb would require some sort of interface. Perhaps a digital nose could send data over Bluetooth to a Neuralink.

This is still an open opportunity and not well answered, in my opinion. We really need more research showing how artificial olfaction can be co-trained with vision and language AI models to get a better understanding. For example, if I have two apples (one wax apple and one real apple), current AI models cannot tell me which one is real. But an AI model integrated with an olfaction sensor could because fruits emit chemicals. The ability to give data to an AI and say “from where is this aroma coming” or “go find the source of compound X” poses big opportunities for multimodal learning among vision, audio, and olfaction.

Yes, I was first author on this paper. I (Kordal) worked under Dr Shalini Prasad for this.

Yes, any drug emitting VOCs can, in theory, be detected. Even drugs that do not emit VOCs can probably be detected too, because they are usually mixed with or exposed to VOC-emitting chemicals to make them cheaper (e.g. fentanyl).

Yes, we assessed which VOCs were correlative to various thoracic conditions and built a breathalyzer and AI to help screen for them in breath.