A world of AI agents

Agentic AI systems are powerful, and they are turning up in more and more applications. Getting the implementation right matters more every year.

But almost all of them run behind a chat box or a command line, which makes it hard to see what is going on under the hood. So we built something else: a way of interacting with an agentic system that makes the agents tangible, the interface intuitive, and the whole thing rather more fun.

▶Demo · 2 min 19 s

We built it during my internship at Lawrence Livermore National Laboratory, as a case study in agentic AI for materials science.

The problem: inspecting 3D-printed metal#

Our team's task was to automate the quality assessment of 3D-printed metal parts.

Metal parts can be printed by a process called Laser Powder Bed Fusion (L-PBF), in which a laser melts and fuses metallic powder together, layer by layer. The parts that come out of it are used for all sorts of things.

Printing does not always go to plan, though. Defects can form inside a part, and a part with defects inside it is brittle, and breaks when it should not. Since you cannot see inside a printed part from the outside, the internal structure has to be imaged with a CT scan, and that scan has to be inspected. It is the step that decides whether the part can be trusted.

It has also been done by hand, by human experts. That is slow and expensive, and it is very often the bottleneck in the whole process.

The interface: a world you can walk into#

The system is a game-like world, in the style of the Pokémon games a lot of us grew up with. You walk into it.

There are rabbits, squirrels and coyotes running around the grass, and buildings you can go into. Walk into the Lab and you find the four agents at their desks.

You start a workflow by talking to an agent directly. They move around the world while they work, so you can see at a glance how the four of them are getting on, and their progress shows up in thought balloons above their heads. Nothing happens out of sight.

The agents#

There are four of them, each named after one of us.

Yu — the guide#

Answers questions about the system and explains how to work with the other three. Somewhere to start when you do not yet know what the place can do.

Andrew — CT inspection#

Visualises the CT scan and finds the defects in it. The scan is presented as an interactive 3D model you can rotate and zoom, with suspected and confirmed defects picked out in different colours, next to the original slices.

Naoki — reporting#

Turns the analysis into dataset-level quality reports, as HTML or PDF. You describe the report you want and it writes that one, rather than one fixed format for every occasion.

Jasper — physical simulation#

Sets up bounded physical simulations to test how strong a part actually is. It plans the study with you in conversation first, then presents a simulation card laying out what it intends to run. Nothing starts until you approve it.

What it is for#

The point of the case study is not the metal. It is that an agentic system does not have to be a chat box.

Giving the agents a place to be, and letting you watch them work in it, makes the system easier to understand and more pleasant to use, and the same idea would carry to agents doing something else entirely, in another field. That is the part worth taking away.

Acknowledgements#

Thank you to my brilliant fellow interns, Naoki Sakai, Andrew Pearce and Jasper Dong. Building this from scratch in under six weeks was only possible because of their talent and their teamwork.

Thank you also to the DSSI leadership and to our mentors, for their support and guidance throughout. We had a great time learning and building together. A fantastic summer.