Generative AI

Source: The Storyteller used inputs a character, a building, and text, combined with LoRAs trained on specific visual styles, to automatically generate a comic strip. MaCAD Generative AI 2024/25, by Students: Christina Christoforou and Renuka Deshpande.

The Digital Tools for Generative AI seminar explores the practical use of generative models within the Architecture, Engineering, and Construction (AEC). The integration of these tools into design practice represents a significant shift in how architects, engineers, and designers develop ideas, visualize proposals, and explore complex spatial problems. Generative AI enables practitioners to rapidly investigate new design possibilities, augment creative workflows, and automate portions of visualization and prototyping that traditionally required extensive manual effort.

Students will engage with current generation techniques across both 2D and emerging 3D. The seminar introduces platforms such as ComfyUI, Google Colab, and Hugging Face Diffusers, allowing participants to run, modify, and experiment with diffusion-based models in various environments. Through these tools, students will learn to generate outputs from inputs through various pipelines that extend from image generation to 3D assets.

The course covers generating images with pre-trained models, fine-tuning models using Low-Rank Adaptation (LoRA) with custom datasets, and adjusting inference parameters to understand how prompt design, sampling methods, and guidance settings influence visual outputs. Students will also explore node-based generative pipelines in ComfyUI, gaining a deeper understanding of how diffusion models and custom workflows can be constructed for specific design tasks. By the end of the seminar, students will synthesise their experiments into interactive interfaces using Gradio, enabling others to explore prompts, parameters, and generated outputs. HuggingFace Pages offers an optional pathway for sharing these interfaces with a broader audience.

The course emphasises both conceptual understanding and practical deployment, equipping participants with the literacy needed to integrate generative AI tools into contemporary computational design workflows.

This course was initially developed and taught by Nono Martínez Alonso, whose work we gratefully acknowledge and hope to build upon.

Learning Objectives

At course completion the student will:

• Learn the history and evolution of ML models from image to emerging 3D generation.
• Understand key concepts of embeddings, latent space, network architecture, denoising, sampling, conditional generation, guidance, training, and fine-tuning.
• Generate images from text prompts, image inputs, and multimodal conditioning.
• Edit, extend, and remix images while maintaining stylistic and spatial coherence.
• Fine-tune models using custom datasets and Low-Rank Adaptation (LoRA) techniques.
• Control image generation using additional inputs such as sketches, edge maps, segmentation masks, and depth maps.
• Utilize node-based workflows in ComfyUI and experiment with models using Google Colab and Hugging Face Diffusers.
• Explore early 3D generative workflows and spatial outputs using diffusion models.
• Utilize batch prompting and parameter variation strategies to systematically iterate.
• Create interactive interfaces using Gradio to present workflows.