Piece by Piece: BrickGPT Assembles Generative Designs

Carnegie Mellon University (CMU) researchers have developed a novel way to prototype new designs using Lego^® bricks. BrickGPT leverages generative AI with robotic motion to stack Lego into some standard configurations, forming a starting point for new designs. The instructions can be used to program a robot or a brick-by-brick guide written for those who want to build Lego creations by hand.

A robot begins stacking Lego to build a prototype. Image used courtesy of CMU

Why Lego?

Lego bricks make the perfect building block for prototyping. Available in many shapes, colors, and configurations, yet completely standardized, they can be stacked in countless ways. They are rigid enough to provide strength for large objects, yet lightweight enough to be passed around a table at a design meeting. The locking mechanism built into each Lego brick allows them to be used thousands of times so that yesterday's prototype can be disassembled and become today's new design.

Generative AI

Generative AI is often used to create new content for writing, art, troubleshooting, and many other purposes. BrickGPT represents the first steps into “generative manufacturing,” where a designer or engineer can start prototyping a new idea quickly. From there, the instruction set can be sent to an automated assembly process, and a full manufacturing process can be implemented.

AI is used to speed up the design process. Instead of the user having to configure the base shape, the generative AI understands template designs to start. A user can start with a “guitar” or “chair” template, and the robot has a starting place to build the next prototype. It can also check on the design throughout the building process to ensure stability.

Physics Engine

At the heart of this system is a complex physics calculator. As BrickGPT builds an object, it constantly evaluates its structural integrity. If the design looks unstable at any point during the build process, the physics engine built into the AI can back up, undoing construction and reconfiguring as needed.

Researchers developed a dataset with over 47,000 bricks to train the AI, which they named StableText2Brick. In StableText2Brick, the researchers created over 28,000 unique 3D objects, each with descriptive captions to aid in the development of larger items.

Two robots work together, choosing the proper Lego brick to build an object. Image used courtesy of CMU

Applications

BrickGPT has the potential for many applications in the near future. After all, who doesn’t like playing with Lego?

There are 21 objects in BrickGPT's library, and the researchers at CMU would like to expand it to include many more items. With an expanded library, customization and prototyping can be performed much more quickly than is conventionally possible.

Also, because the prototypes are made from bricks, they can be disassembled and reused. Designs can be tested and modified without starting from scratch on each iteration. This puts BrickGPT's prototyping at a significant advantage over additive manufacturing, machining, and other such techniques, where a design change means a whole new, potentially expensive prototype must be created.