Learning the Matrix: Rules, Play, and Interpretation in Machine Learning
Photo Credits: ITI TUWien / Zeynep Aksöz Balzar
This experimental workshop investigates how spatial rules can be encoded, interpreted, and reimagined through a matrix-based design system that combines physical play, symbolic abstraction, and generative algorithms. Using Kapla blocks as a tangible grammar of structure, students will formulate construction rules—based on orientation, adjacency, support, and repetition—and encode them within colour-coded matrices. These matrices serve as both visual languages and machine-readable blueprints, in which each colour carries meaning and spatial logic.
Throughout the course, the matrix functions as a semiotic surface: a compressed 2D representation of 3D spatial rules, material constraints, and assembly possibilities. Students will explore how their rule-based systems translate into this matrix form, and how such abstractions can be used as training data for Generative Adversarial Networks (GANs). The goal is not to generate finished designs, but to prompt unexpected variations—emerging from the machine’s interpretation of human-devised structures.
The human designer plays an essential role in this loop, not only by defining the initial rule set but also by interpreting the outputs: curating, simulating, and re-evaluating the matrices generated by the GAN. In this way, the course foregrounds interpretation as a design act, positioning AI as a co-author whose contributions require critical and creative human response.
The final phase of the course involves robotic construction: students will simulate and test various GAN-generated matrices for structural stability and semantic fidelity, select one, and build it using robotic pick-and-place routines. This rematerialization process completes the cycle from embodied rule-making to digital abstraction, from AI interpretation back into physical form.
Learning Objectives
KEYWORDS: machine learning, matrices, creative coding, computational design
Rethinking Towe Typology with AI: This project explores the redesign of contemporary tower structures using machine learning (ML), challenging the repetitive, profit-driven design seen in cities like Hong Kong. The goal is to discover new typologies through AI and stacking rules, and tren translate them to a robotic fabrication system (pick and place). AI-generated towers … Read more
| INTRODUCTION | Learning Structures: From Parametric Rules to Machine-Made Forms. In this project, we explored the interplay between parametric design, machine learning, and robotic fabrication. Starting from Kapla block assemblies controlled by simple deformation rules, we trained a GAN to reinterpret and generate new structural variations. The process concluded with robotic pick-and-place construction, closing … Read more
A Closed-Loop RL-GAN-ML Pipeline for Generative Design VIDEO: Objective: Demonstrate an intelligent system for stable Kapla stack design, showcasing its intelligent, iterative pipeline. Automated Generation of Stable Kapla Plank Stacks. 1. Introduction The Python script (`loop.py`) presents a sophisticated framework for automating the design of stable Kapla plank stacks, a wooden construction toy consisting of … Read more
