Mars-DL

Starting point / motivation

Planetary robotic missions contain vision instruments for various mission-related and science tasks such as 2D and 3D mapping, geologic characterization, atmospheric investigations, or spectroscopy for exobiology. One major application for computer vision is the characterization of scientific context, and the identification of scientific targets of interest (regions, objects, phenomena) for being investigated by other scientific instruments. Due to high variability of appearance of such potentially scientific targets it requires well-adapted yet flexible techniques, one of them being Deep Learning.

Contents and goals

Mars-DL therefore targets in the adaptation and test of simulation & deep learning mechanisms for autonomous detection of scientific targets of interest during robotic planetary surface missions. So far science autonomy has been addressed only recently with increased mobility on planetary surfaces and the upcoming need for planetary rovers to react autonomously to given science opportunities, well in view of limited data exchange resources and tight operations cycles. Machine learning and in particular Deep Learning is a technique used in computer vision to recognize content in images, categorize it and find objects of specific semantics.

Methods

In its default workflow it requires large sets of training data with known / manually annotated objects, regions or semantic content. Within Mars-DL, training uses different approaches, namely

• using existing planetary images from past and present missions including relevant field campaigns (MER, MSL, ExoFit),
• using known targets from existing catalogues,
• using simulation by virtual placement of known targets in true context environment.

Training and validation will explore the possibility to search scientifically interesting targets across different sensors, investigate the usage of different cues such as 2D (multispectral / monochrome) and 3D, as well as spatial relationships between image data and regions thereon. For past and present missions the project will help explore & exploit further existing millions of planetary surface images that still hide undetected science opportunities.

The project will constitute an important application and demo case for Austria’s capability to contribute to international mission-related scientific Planetary Exploration. Mars-DL activities will build upon JR & VRVis’ expertise in planetary vision processing and data products provision for scientific use, SLR’s operational framework for deep-learning based industrial inspection, and NHM’s background on planetary science.

The exploratory project will assess the feasibility of machine-learning based support during and after missions by automatic search on planetary surface imagery to raise science gain, meet serendipitous opportunities and speed up the tactical and strategic decision making during mission planning.

Expected results

An automatic"Science Target Consultant" (STC) is realized in prototype form which, as a test version, can be plugged in to ExoMars operations once the mission has landed. During mission operations of forthcoming missions (Mars 2020, and ExoMars & SFR in particular) the STC can help avoid the missing of opportunities that may occur due to tactical time constraints preventing in-depth check of image material.

In case the proposed principles can be successfully demonstrated, the target of a follow-on activity will be a working prototype of the STC, deployed to mission operations of ExoMars 2020. Terrestrial applications mainly in the geology domain will be explored as well.

Coordinator

JOANNEUM RESEARCH Forschungsgesellschaft mbH

Project partner

• Natural History Museum Vienna
• SLR Engineering GMBH
• VRVis Zentre for Virtual Reality and Visualisation

JOANNEUM RESEARCH Forschungsgesellschaft mbH
Dipl.-Ing. Gerhard Paar
Steyrergasse 17
A-8010 Graz