Florian Jaensch

Herr M.Sc.

Gruppenleiter "Virtuelle Methoden in der Produktionstechnik"
Institut für Steuerungstechnik der Werkzeugmaschinen und Fertigungseinrichtungen

Kontakt

+49 711 685 82532
+4971168572532
E-Mail
Visitenkarte (VCF)

Seidenstr. 36
70174 Stuttgart
Deutschland
Raum: 4.065

Fachgebiet

Künstliche Intelligenz für die Steuerungstechnik
Reinforcement Learning am Digitalen Zwilling
Hardware-in-the-Loop Simulation
Virtuelle Inbetriebnahme

Publikationen

Veröffentlichungen:

2022
1. Kienzlen, Annika ; Jaensch, Florian ; Verl, Alexander ; Cheng, Leo: Concept for a Reinforcement Learning Approach to Navigate Catheters Through Blood Vessels. In: 2022 28th International Conference on Mechatronics and Machine Vision in Practice (M2VIP), 2022 28th International Conference on Mechatronics and Machine Vision in Practice (M2VIP), 2022
  - BibTeX
  - Link
  BibTeX
  @inproceedings{kienzlen2022concept, author = {Kienzlen, Annika and Jaensch, Florian and Verl, Alexander and Cheng, Leo}, booktitle = {2022 28th International Conference on Mechatronics and Machine Vision in Practice (M2VIP)}, doi = {10.1109/M2VIP55626.2022.10041096}, eventdate = {16-18 Nov. 2022}, eventtitle = {2022 28th International Conference on Mechatronics and Machine Vision in Practice (M2VIP)}, title = {Concept for a Reinforcement Learning Approach to Navigate Catheters Through Blood Vessels}, url = {https://doi.org/10.1109/M2VIP55626.2022.10041096}, year = 2022 }
  Link
  https://doi.org/10.1109/M2VIP55626.2022.10041096
2. Jaensch, Florian ; Kübler, Karl ; Schwarz, Elmar ; Verl, Alexander: Test-Driven Reward Function for Reinforcement Learning: A Contribution towards Applicable Machine Learning Algorithms for Production Systems. In: Procedia CIRP, Procedia CIRP. Bd. 112 (2022), S. 103–108. — 15th CIRP Conference on Intelligent Computation in ManufacturingEngineering, 14-16 July 2021
  - BibTeX
  - Link
  BibTeX
  @article{JAENSCH2022103, abstract = {Reinforcement Learning algorithms find more and more application in fields where complex tasks need to be solved. The automation of production systems is one of those fields. Normally, programming a control system defines the automation strategy. Previous contributions by the authors have shown that a so-called agent can learn automation strategies for production systems using a Reinforcement Learning setup. However, the development of the reward function for the agent can be challenging and needs Reinforcement Learning domain knowledge. This paper introduces a novel approach in combining Test-Driven Development with Reinforcement Learning in order to solve the problem of a suitable reward function. In the presented approach predefined test cases are used to derive rewards for the agent. The use of an automated test framework allows for continuous learning sequences until all test cases are passed. An application example of a robot cell is used to demonstrate the novel approach and verify its suitability and usability. The first application shows promising results for further examination towards more application scenarios.}, author = {Jaensch, Florian and Kübler, Karl and Schwarz, Elmar and Verl, Alexander}, doi = {https://doi.org/10.1016/j.procir.2022.09.043}, issn = {2212-8271}, journal = {Procedia CIRP}, note = {15th CIRP Conference on Intelligent Computation in ManufacturingEngineering, 14-16 July 2021}, pages = {103-108}, title = {Test-Driven Reward Function for Reinforcement Learning: A Contribution towards Applicable Machine Learning Algorithms for Production Systems}, url = {https://www.sciencedirect.com/science/article/pii/S2212827122011970}, volume = 112, year = 2022 }
  Link
  https://www.sciencedirect.com/science/article/pii/S2212827122011970
3. von Bergen, Timm ; Hert, Christian ; Röck, Sascha ; Klingel, Lars ; Jaensch, Florian ; Verl, Alexander: Simulationsgestützter Betrieb von Indoor-Flugrobotern. In: wt Werkstattstechnik online, wt Werkstattstechnik online. Bd. 112, VDI Fachmedien GmbH and Co. KG (2022), Nr. 05, S. 309–313
  - BibTeX
  - Link
  BibTeX
  @article{vonbergen2022simulationsgesttzter, author = {von Bergen, Timm and Hert, Christian and Röck, Sascha and Klingel, Lars and Jaensch, Florian and Verl, Alexander}, doi = {doi.org/10.37544/1436-4980-2022-05}, journal = {wt Werkstattstechnik online}, number = 05, pages = {309-313}, publisher = {VDI Fachmedien GmbH and Co. KG}, title = {Simulationsgestützter Betrieb von Indoor-Flugrobotern}, url = {https://elibrary.vdi-verlag.de/10.37544/1436-4980-2022-05/wt-werkstattstechnik-online-jahrgang-112-2022-heft-05}, volume = 112, year = 2022 }
  Link
  https://elibrary.vdi-verlag.de/10.37544/1436-4980-2022-05/wt-werkstattstechnik-online-jahrgang-112-2022-heft-05
2021
1. Wnuk, Markus ; Jaensch, Florian ; Tomzik, David Adrian ; Chen, Zixiao ; Terfurth, Jonathan ; Kandasamy, Sivakumar ; Shahabi, Jalil ; Garrett, Amy ; u. a.: Challenges in Robotic Soft Tissue Manipulation—Problem Identification Based on an Interdisciplinary Case Study of a Teleoperated Drawing Robot in Practice. In: Billingsley, J. ; Peter, B. (Hrsg.) ; Billingsley, J. ; Peter, B. (Hrsg.): . Bd. Mechatronics and Machine Vision in Practice 4 : Springer, 2021 — ISBN 978-3-030-43702-2, S. 245–262
  - BibTeX
  - Link
  BibTeX
  @inbook{wnuk2021challenges, abstract = {Manipulating soft materials with robots is a complex and highly interdisciplinary research topic. Due to the underlying body of involved theories including material science, (bio-)mechanics, control, sensing as well as electrical and mechanical engineering, it is even difficult to provide a comprehensive problem formulation. In this paper, the problem of identifying challenges within the different research areas and between their respective interfaces is addressed with a practical case study. Based on a generalized sandbox scenario of a remotely controlled drawing robot, challenges with regard to the interdisciplinary research fields of robot control, architecture, sensor development, material modelling and interaction between rigid actuators acting on soft materials are investigated and analyzed. The major contribution of this paper is the explicit identification of unresolved research problems in the field and a quantification of the correlations between the involved disciplines. Furthermore, novel approaches on how to circumvent some of the found challenges in the future are proposed.}, author = {Wnuk, Markus and Jaensch, Florian and Tomzik, David Adrian and Chen, Zixiao and Terfurth, Jonathan and Kandasamy, Sivakumar and Shahabi, Jalil and Garrett, Amy and Mahmoudinezhad, Masoumeh Hesam and Csiszar, Akos and Xu, Weiliang and Röhrle, Oliver and Verl, Alexander}, doi = {https://doi.org/10.1007/978-3-030-43703-9_20}, editor = {Billingsley, John and Peter, Brett}, isbn = {978-3-030-43702-2}, pages = {245-262}, publisher = {Springer}, title = {Challenges in Robotic Soft Tissue Manipulation—Problem Identification Based on an Interdisciplinary Case Study of a Teleoperated Drawing Robot in Practice.}, url = {https://link.springer.com/chapter/10.1007/978-3-030-43703-9_20}, volume = {Mechatronics and Machine Vision in Practice 4}, year = 2021 }
  Link
  https://link.springer.com/chapter/10.1007/978-3-030-43703-9_20
2. Heiland, Stefan ; Klingel, Lars ; Penter, Lars ; Jaensch, Florian ; Schenke, Christer ; Ihlenfeldt, Steffen ; Verl, Alexander: Virtual Tool Commissioning using LS-DYNA Functional Mock-up Interface. In: , 2021
  - BibTeX
  - Link
  BibTeX
  @conference{heiland2021virtual, author = {Heiland, Stefan and Klingel, Lars and Penter, Lars and Jaensch, Florian and Schenke, Christer and Ihlenfeldt, Steffen and Verl, Alexander}, eventdate = {2021}, eventtitle = {13th European LS-DYNA Conference 2021, Ulm, Germany}, title = {Virtual Tool Commissioning using LS-DYNA Functional Mock-up Interface}, url = {https://www.dynalook.com/conferences/13th-european-ls-dyna-conference-2021/simulation-methods/heiland_tu_dresden.pdf}, year = 2021 }
  Link
  https://www.dynalook.com/conferences/13th-european-ls-dyna-conference-2021/simulation-methods/heiland_tu_dresden.pdf
3. Jaensch, Florian ; Steidle, Adrian ; Verl, Alexander: Curriculum Multi-Stage Reinforcement Learning for Automated Interlinked Production Systems on Virtual Commissioning Simulations. In: 2021 Third International Conference on Transdisciplinary AI (TransAI), 2021 Third International Conference on Transdisciplinary AI (TransAI), 2021, S. 129–136
  - BibTeX
  BibTeX
  @inproceedings{9565636, abstract = {In order to automate the software engineering process of interlinked production systems, reinforcement learning applications can be used to learn the control flow logic on the basis of virtual production systems. Since the simulation- based prototypes are available for virtual commissioning (VC) anyway, they can be used simultaneously as reinforcement learning environments. In this work, the event-discrete flow logic for the transport and assembly of a target workpiece is learned automatically by reinforcement learning on the real use case of the VC simulation of a PLC-based production system. According to the idea of curriculum learning, the system is trained separately in subsystems to support its modularity and to reduce the complexity of the overall learning process. With regard to the learning processes, subsystems, sequence errors, termination criteria and necessary action and state adjustments typical for the PLC-based plant are identified and implemented in the VC simulation. The reward functions are derived with respect to the individual subsystems. The learned controls of the subsystems are then merged back together for a complete flow of the entire system.}, author = {Jaensch, Florian and Steidle, Adrian and Verl, Alexander}, booktitle = {2021 Third International Conference on Transdisciplinary AI (TransAI)}, doi = {10.1109/TransAI51903.2021.00031}, month = {09}, pages = {129-136}, title = {Curriculum Multi-Stage Reinforcement Learning for Automated Interlinked Production Systems on Virtual Commissioning Simulations}, year = 2021 }
2020
1. Jaensch, Florian ; Verl, Alexander: Virtual Commissioning Simulation as Reinforcement Learning Environment for Robot Cable Handling. In: 2020 Third International Conference on Artificial Intelligence for Industries (AI4I), 2020 Third International Conference on Artificial Intelligence for Industries (AI4I), 2020, S. 27–31
  - BibTeX
  BibTeX
  @inproceedings{9253103, abstract = {The paper introduces an approach to generate control strategies for cable handling tasks automatically, using Reinforcement Learning. Cable handling tasks require human like sensomotoric and cognitive abilities. Humans develop an understanding of the cable behaviour and are able to derive control strategies, how to handle the cable. Transferring these strategies to an industrial robot control system with the usual programming interface is very difficult. The introduced approach addresses this problem by giving the robot control the possibility to learn the cable handling strategies by itself. Therefore an Virtual Commissioning simulation environment was adapted to model Reinforcement Learning problems using the OpenAI Gym and Functional Mock-Up Interface standard.}, author = {Jaensch, Florian and Verl, Alexander}, booktitle = {2020 Third International Conference on Artificial Intelligence for Industries (AI4I)}, doi = {10.1109/AI4I49448.2020.00013}, month = {09}, pages = {27-31}, title = {Virtual Commissioning Simulation as Reinforcement Learning Environment for Robot Cable Handling}, year = 2020 }
2019
1. Jaensch, Florian ; Csiszar, Akos ; Sarbandi, Janik ; Verl, Alexander: Reinforcement Learning of a Robot Cell Control Logic using a Software-in-the-Loop Simulation as Environment. In: 2019 Second International Conference on Artificial Intelligence for Industries (AI4I), 2019 Second International Conference on Artificial Intelligence for Industries (AI4I), 2019, S. 79–84
  - BibTeX
  - Link
  BibTeX
  @inproceedings{9027783, abstract = {This paper introduces a method for automatic robot programming of industrial robots using reinforcement learning on a Software-in-the-loop simulation. The focus of the the paper is on the higher levels of a hierarchical robot programming problem. While the lower levels the skills are stored as domain specific program code, the combination of the skills into a robot control program to solve a specific task is automated. The reinforcement learning learning approach allows the shopfloor workers and technicians just to define the end result of the manufacturing process through a reward function. The programming and process optimization is done within the learning procedure. The Software-in-the-loop simulation with the robot control software makes it possible to to interpret the real program code and generate the exact motion. The exact motion of the robot is needed in order to find not just an optimal but also a collision-free policy.}, author = {Jaensch, Florian and Csiszar, Akos and Sarbandi, Janik and Verl, Alexander}, booktitle = {2019 Second International Conference on Artificial Intelligence for Industries (AI4I)}, doi = {10.1109/AI4I46381.2019.00027}, month = {09}, pages = {79-84}, title = {Reinforcement Learning of a Robot Cell Control Logic using a Software-in-the-Loop Simulation as Environment}, url = {https://ieeexplore.ieee.org/document/9027783/}, year = 2019 }
  Link
  https://ieeexplore.ieee.org/document/9027783/
2018
1. Jaensch, Florian ; Csiszar, Akos ; Kienzlen, Annika ; Verl, Alexander: Reinforcement Learning of Material Flow Control Logic Using Hardware-in-the-Loop Simulation. In: 2018 First International Conference on Artificial Intelligence for Industries (AI4I), 2018 First International Conference on Artificial Intelligence for Industries (AI4I), 2018, S. 77–80
  - BibTeX
  - Link
  BibTeX
  @inproceedings{8665712, abstract = {In this paper the concept of reinforcement learning agent is presented, which can deduce the correct control policy of a plant by acting in its digital twin (the HiL simulation). This way the agent substitutes a real control system. By using reinforcement learning methods, a proof of concept application is presented for a simplistic material flow system, with the same type of access to the digital twin which a PLC controller-hardware would have. With the presented approach the agent is able to find the correct control policy.}, author = {Jaensch, Florian and Csiszar, Akos and Kienzlen, Annika and Verl, Alexander}, booktitle = {2018 First International Conference on Artificial Intelligence for Industries (AI4I)}, doi = {10.1109/AI4I.2018.8665712}, month = {09}, pages = {77-80}, title = {Reinforcement Learning of Material Flow Control Logic Using Hardware-in-the-Loop Simulation}, url = {https://ieeexplore.ieee.org/document/8665712/}, year = 2018 }
  Link
  https://ieeexplore.ieee.org/document/8665712/
2. Klein, Maike ; Hoher, Simon ; Kimpeler, Simone ; Lehner, Maximilian ; Jaensch, Florian ; Kühfuß, Fabian ; Lehmann, Hagen ; Snelting, Femke: Machines Without Humans—Post-Robotics. In: Coeckelbergh, M. ; Loh, J. ; Funk, M. (Hrsg.) ; Coeckelbergh, M. ; Loh, J. ; Funk, M. (Hrsg.): Proceedings of Robophilosophy 2018 / TRANSOR 2018, Proceedings of Robophilosophy 2018 / TRANSOR 2018, 2018, S. 88–92
  - BibTeX
  BibTeX
  @inproceedings{klein2018machines, author = {Klein, Maike and Hoher, Simon and Kimpeler, Simone and Lehner, Maximilian and Jaensch, Florian and Kühfuß, Fabian and Lehmann, Hagen and Snelting, Femke}, booktitle = {Proceedings of Robophilosophy 2018 / TRANSOR 2018}, doi = {10.3233/978-1-61499-931-7-88}, editor = {Coeckelbergh, Mark and Loh, Janina and Funk, Michael}, pages = { 88 - 92}, title = {Machines Without Humans—Post-Robotics}, year = 2018 }
3. Jaensch, Florian ; Csiszar, Akos ; Scheifele, Christian ; Verl, Alexander: Digital Twins of Manufacturing Systems as a Base for Machine Learning. In: 2018 25th International Conference on Mechatronics and Machine Vision in Practice (M2VIP), 2018 25th International Conference on Mechatronics and Machine Vision in Practice (M2VIP), 2018, S. 1–6
  - BibTeX
  - Link
  BibTeX
  @inproceedings{8600844, abstract = {In the engineering phase of modern manufacturing systems, simulation-based methods and tools have been established to face the increasing demands on time-efficiency and profitability. In the application of these simulation solutions, model-based digital twins are created, as multi-domain simulation models to describe the behavior of the manufacturing system. During the production process, a data-driven digital twin arises in the context of industry 4.0 based on an increasing networking and new cloud technologies. Recent developments in machine learning of fer new possibilities in conjunction with the digital twin. These range from data-based learning of models to learning control logic of complex systems. This paper proposes a combined model-based and data-driven concept of a digital twin. It shows how to use machine learning in connection with these models, in order to archive faster development times of manufacturing systems.}, author = {Jaensch, Florian and Csiszar, Akos and Scheifele, Christian and Verl, Alexander}, booktitle = {2018 25th International Conference on Mechatronics and Machine Vision in Practice (M2VIP)}, doi = {10.1109/M2VIP.2018.8600844}, month = {11}, pages = {1-6}, title = {Digital Twins of Manufacturing Systems as a Base for Machine Learning}, url = {https://ieeexplore.ieee.org/document/8600844/}, year = 2018 }
  Link
  https://ieeexplore.ieee.org/document/8600844/
2017
1. Csiszar, Akos ; Jaensch, Florian ; Kienzlen, Annika ; Scheifele, Christian: Machine Learning in Steuerungstechnik und Robotik. In: SPS MAGAZIN, SPS MAGAZIN. Bd. 30 (2017), Nr. 12 2017, S. 42--44
  - BibTeX
  BibTeX
  @article{Csiszar.2017, author = {Csiszar, Akos and Jaensch, Florian and Kienzlen, Annika and Scheifele, Christian}, journal = {SPS MAGAZIN}, number = {12 2017}, pages = {42--44}, title = {Machine Learning in Steuerungstechnik und Robotik}, volume = 30, year = 2017 }
2. Jaensch, Florian ; Neyrinck, Adrian ; Lechler, Armin ; Verl, Alexander: Variantenreiche Fabrikplanung: Automatische Variantenerzeugung für den simulationsbasierten Vergleich von Anlagenprojektierungen. In: wt Werkstattstechnik online, wt Werkstattstechnik online. Bd. 107 (2017), Nr. 9, S. 640–646
  - BibTeX
  BibTeX
  @article{Jaensch.107, author = {Jaensch, Florian and Neyrinck, Adrian and Lechler, Armin and Verl, Alexander}, journal = {wt Werkstattstechnik online}, number = 9, pages = {640-646}, title = {Variantenreiche Fabrikplanung: Automatische Variantenerzeugung f{\"u}r den simulationsbasierten Vergleich von Anlagenprojektierungen}, urldate = {24.10.2017}, volume = 107, year = 2017 }

Lehre

Betreuung des Praktikums Hardware-in-the-Loop Simulation einer Werkzeugmaschine
Betreuung der Bachelorprojektarbeit Aufbau von Antrieben, Steuerung und HMI einer Werkzeugmaschine
Betreuung von Bachelor-, Studien- und Masterarbeiten

Florian Jaensch

Kontakt

Fachgebiet

Publikationen

2022

BibTeX

Link

BibTeX

Link

BibTeX

Link

2021

BibTeX

Link

BibTeX

Link

BibTeX

2020

BibTeX

2019

BibTeX

Link

2018

BibTeX

Link

BibTeX

BibTeX

Link

2017

BibTeX

BibTeX

Lehre

Zielgruppe

Formalia

Services

Organisation

Florian Jaensch

Kontakt

Fachgebiet

Publikationen

2022

BibTeX

Link

BibTeX

Link

BibTeX

Link

2021

BibTeX

Link

BibTeX

Link

BibTeX

2020

BibTeX

2019

BibTeX

Link

2018

BibTeX

Link

BibTeX

BibTeX

Link

2017

BibTeX

BibTeX

Lehre

So erreichen Sie das ISW

Zielgruppe

Formalia

Services

Organisation