Today’s AI sucks at math, but developing cars and planes with it is still a good idea
Today’s AI sucks at math, but developing cars and planes with it is still a good idea
The future of engineering is assisted. And it will lead to better and (!) safer products
TL;DR
Given that ChatGPT likes to invent and imagine things, introducing AI Assisted Hardware Engineering sounds like a recipe for disaster. Why use an unpredictable AI in the design of planes, cars and rockets; where being exact matters so much?
By diving into actual hardware development tasks that engineers have to perform regularly and looking at concrete AI generated engineering artifact examples it becomes clear that Assisted Engineering will lead to faster and better design of safer products; always as long as we understand its limitations and how you have to pair current technologies with math-capabilities to make engineers 10x faster but their designs still reliable and safe.
Predicting the future of Engineering
Background
As an aerospace engineer who has worked in the past on one of Europe's biggest Satellite Programs (MTG) and as the CEO of an engineering software company that has helped hundreds of modern companies digitize their hardware engineering, I am often asked about what the future of engineering will look like.
While having discussed broad ideas in the past, December 2022 has been a pivotal point to shaping my opinion: I am now convinced that the future of engineering is assisted.
What changed?
At the latest with the release of ChatGPT, it became clear that Large Language Models (LLMs) are able to suggest, complete, improve and summarize text extraordinarily well. While thousands of articles have been written about the pros and cons and possible implications, I want to focus solely on how hardware engineering will be affected by this technological shift.
In this article I will refer to engineering features that are enabled through AI and more specifically through so-called Large Language Models (LLMs) such as ChatGPT as “Assisted Engineering”.
Today: How we develop cars, ships, planes, rockets and most other physical goods
If you are a daily practicing engineer in an industrial setting, feel free to jump to the future section.
V-Model and Documents
Hardware engineering in general follows the so-called “V-Model”. Without going too much into detail, the key take-away is that in principle it can be simplified into a 5 step process:
Specify: You start by formally specifying your design (“How fast do I want the car to be able to drive?”)
Break-Down: then you break these needs down into requirements for subsystems and parts (“How much torque does the motor need to generate for that?”).
Design: Next you work on a design that can fulfill those requirements and is optimized for cost / quality / time (“Which battery do I use for the car and what is the distance the car can travel with it?”)
Verify: You now verify subsystems and components through analyses and tests (“How many RPMs do I get when I put the motor on a teststand?”)
Validate: In the end you validate the fully integrated product; ideally with a customer (“Let’s drive this car and see how fast and far it goes.”)
There can be dozens or thousands of engineers involved in the above steps; depending on size and complexity of the product involved.
Today the development of products takes place in specialized software (CAD for physical design, Simulation Software, etc.). There have been approaches in the past decades to manually build up models (see SysML), but in the day-to-day life of most engineers in industry the communication between engineers and teams is mostly document-based (Requirement Specifications, Interface Documents, etc.).
How is product safety ensured today?
Safety from a product-design perspective is mainly achieved using 2 key mechanisms:
Products are developed according to standards. These are lists of requirements that describe properties of the product itself, the process of developing as well as the accepted methods for testing/verifying it. Sometimes these standards come from external agencies and sometimes these are company-internal standards.
In certification and qualification processes, the company has to prove that the product and processes used to develop it indeed have followed the standards.
When it comes to the certification process a rule of thumb is that the more dangerous failures of products are (think cars, planes, etc. vs. smart watches), the heavier the certification process is to prove that the design was done correctly.
This means that while engineers are always responsible to develop good products, engineering as a discipline does not rely on single engineers not making mistakes.
Today this means that engineers might make e.g. a mistake in their spreadsheet that calculates critical properties of a product, but through a process of design reviews and qualification tests this single mistake will be caught before it goes into production.
This is very important to be understood as we assess later whether besides spreadsheets and documents an AI assistant can also be used to derive critical engineering elements.
The Future™
Every engineering action will be assisted. What this means concretely is that the following processes won’t be done by a single person anymore, but by a person or team collaborating with an AI.
Let’s dive into what this looks like concretely with technology that is available already since the beginning of 2023:
Going from 0 to 1
Lots of engineering work consists in coming up with solutions to novel problems. However, staring at a blank page is as much a problem for creative engineering work as it is for writers of novels. It is much easier to start from a draft that you edit to your needs than to try thinking of everything yourself.
Assisted Engineering allows you to brainstorm within seconds a starting point to many potential aspects. Consider that you are e.g. tasked to design the landing gear of a drone. Instead of trying to think of all the aspects that might be relevant to this piece of equipment you can start discovering which aspects your solution needs to cover:
And suddenly we are ready to think about lots of aspects. Not only about the landing gear itself, but also to think about drag forces, failure cases, material-choices, shock, monitoring and risk management.
These requirements are far from being good or final, but within seconds we are in the middle of thinking about the detailed design. The Results have accelerated us and we can now start diving deeper into multiple aspects:
And here we are. Within minutes we have reached the point in which we are able to consider and formulate requirements that are testable.
Do these requirements need work and refinement, before sending them out to a supplier? Absolutely. Do the values need to be thought through and checked with other stakeholders? Definitively. Did we arrive here 10x as fast as we would have without Assisted Engineering? You bet.
Retrieving existing information
Engineering work also contains a non-trivial amount of trying to find the correct information that is needed at a given point for decision making. These may be values from datasheets, requirements, verification methods, etc.
Structured engineering data, such as in PLM and EIM tools is already helpful, since data exists in these tools in machine readable format instead of PDFs and other documents. However, Assisted Engineering capabilities will allow engineers to ask questions to retrieve data easily, both when interacting alone in front of a PC or when enhancing a conversation in a meeting room.
Imagine that an engineering team is considering using a Raspberry Pi for an early prototype for a sensor in a foundry that they are developing. When someone brings up the fact that the ambient temperature might be too high for the electronics to survive, the pace of the meeting can continue high, if instead of it being interrupted by people looking through documentation you can just ask during the meeting:
Triggering Simulations from text
But of course not all of engineering is based on text alone: engineering requires parameters, formulas, physical units and their interactions to derive meaningful designs. But even in this case, assisted engineering capabilities will allow the interactions to be sped up significantly.
Once you combine natural text input with access to data-driven engineering and simulation tools you can on the fly generate accurate answers for hypothetical scenarios. For me, Valispace’s video (from 2018!) visualizes best what such an interaction can look like and why the difference in getting an answer changes how fast your engineering can advance.
Large language models are (with good prompt engineering) already great today at translating human intent into parameterized functions.
As long as the engineering data is now stored in a collaborative engineering environment with calculation engines, Assisted Engineering can provide accurate answers to loosely formulated questions.
Reviewing
Reviewing the work of other engineers under a specific view-point is an essential every-day task. Many of those aspects rely on knowledge, experience and clear system understanding of the engineer. However there are also many reviewing activities that solely focus on finding duplicates, bad descriptions, unintentional inconsistencies and much more. All of those can be automated to speed up the engineering process significantly and to reduce costly mistakes before even starting the manufacturing process.
For this example I have on purpose introduced a few mistakes into the design description of the Lilium Jet from their website, which an Assisted Engineering system can spot right away:
Similar functionalities will be able to review documentation, specifications, test procedures vs. design, etc. etc.
System Modeling
Designing a product is obviously not only something that happens within the requirements space, but at some point an actual system needs to be described with numbers and models.
Assisted Engineering will be able to extract the structure and initial model-information from text alone.
Being able to convert human language into the initial version of system models removes the entry barriers to model based design that classical approaches have been struggling with.
An Assisted Engineering model that then is specifically trained to suggest relevant formulas in combination with mathematical optimization algorithms is then able to generate meaningful modeling results within a very short time.
Documenting
Documenting a hardware product’s properties and behaviors is an essential part of an engineer’s work, even if not one that engineers usually enjoy a lot. Documentation might be customer-facing (such as user manuals), internal documentation for fellow team members or documents that summarize the state of verification to a certifying body.
In all cases documents have to be carefully reviewed before being released but their preparation and keeping them up-to-date won’t be time consuming anymore at all in an Assisted Engineering environment:
While not being an extraordinary summary, it is a human readable paragraph that contains the key information and has been generated within seconds.
As engineering iterations are becoming shorter and products contain more variants, Assisted Engineering is the best way of allowing engineers to focus on the actual engineering work and be assisted by a never tiring AI.
The risks of using AI (specifically LLMs) for hardware engineering
What LLMs should be used for in Engineering
Hardware engineering is one of life’s areas in which you don’t want companies to make mistakes. You want to be sure that the car is safe, that the bridge has been calculated using the correct forces and that whoever gave the go-ahead for passengers to board a new plane is sure that it will land safely again.
This makes AI generated content in the engineering process the “Best place to start & worst place to end”. Just like a researcher should not treat a wikipedia article as a reliable source to be cited in her final paper; it would be a waste of time not to read up on new concepts that are new to her on wikipedia first.
Concretely that means that generating initial design-outlines, brainstorming solutions as well as summarizing and expanding existing work all are tasks that the engineer can be perfectly assisted with. The AI does not release the engineer from the responsibility of her work but is another tool under her belt, just like spreadsheets, simulation software or other tools have been in the past.
How to complement AI with math and engineering structure?
Language is just one piece of the engineering puzzle. Designing products well relies heavily on values being correctly derived and calculated; simulations to use the right in- and output to calculate performance parameters and boundary conditions and requirements to constantly be checked.
These kinds of problems are “math problems” and should be treated like that: AI (at least the kinds available in early 2023) are not built for and not suited for this kind of task. So the most important step to be taken by any Assisted Engineering software is to allow users to move from the “text” to the “numbers” domain and back easily:
Numbers and physical units that appear in AI generated text need to be converted into parameters that can be linked with formulas amongst each other. Optimization should be left to mathematical optimization algorithms that can produce amazing results (both to calculate optimal parameters as well as to generating shapes) without the need for neural nets.
How to ensure that AI assisted products are safe?
The answer to this question turns out to be quite simple. Even today we don’t trust a single engineer to do their work “correctly”. We have built systems of verification and certification to ensure that the final product is safe.
AI can be used to help engineers derive their designs faster. The same way we spot an excel-formula mistake or a documentation copy-paste error we are as likely as to find problems that have been introduced by the AI.
Making sure that our calculations and numbers are derived from well understood formulas and mathematical algorithms over black-box neural networks allows us to make use of both the creative power of a smart digital engineering colleague who assists us in building better products.
Conclusions
In Assisted Engineering we should treat AI Assistants as our favorite colleagues in the room: They have a lot of creativity and help us derive design solutions faster. They do not get tired of performing tedious and repetitive tasks for us. They have a great memory and their insights bring design tasks forward 10x faster than if they were not in the room. Since they joined our company we have been able to iterate faster; make less mistakes and come up with better and cheaper designs, leading us to faster time to market.
But just like with any other engineer, we would not trust them and get rid of our quality control or certification process; because they might very well make mistakes as well.
Personal plug:
Given my enthusiasm for how the future of work will be Assisted, it won’t come to surprise that at Valispace we just released Assisted Engineering functionalities that catapult hardware developers into this exciting near-term future today.