MaCAD 23/24 Generative AI

GENERATIVE AI

Digital Tools for Generative AI Seminar

The Digital Tools for Generative Planning seminar focuses on the practical applications of Image Synthesis and Language Models within the Architecture, Engineering, and Construction (AEC) industry. Integrating these tools into the design practice represents a pivotal moment for professionals to redefine traditional design processes. Today, architects, engineers, and designers can leverage generative AI to explore new design possibilities, streamline workflows, and address complex challenges in unprecedented ways.

Source: DALL-E 3

In the first half of the seminar, students will learn about state-of-the-art image generation and editing techniques with Image Diffusion Models—how they produce high-quality images from text descriptions and their application to conceptual design and architectural visualization. We’ll generate visuals with existing models, use Low-Rank Adaptation (LoRA) for efficient model fine-tuning with custom datasets, and experiment with model parameters to understand their impact on image outputs.

During the second half of the course, students will explore how Large Language Models (LLMs) can enhance various aspects of architectural and design processes, from generating design concepts to assisting in project development tasks. Through a combination of theoretical discussions and hands-on exercises, participants will learn how to create custom datasets, utilise embeddings and vectorization techniques, design system prompts, create Retrieval-Augmented Generation (RAG) pipelines, and integrate custom LLMs into their design workflows efficiently.

Learning Objectives

• Learn about the history of Machine Learning Image Generation methods and their applications to conceptual design and architectural visualisation.
• Understand key concepts of the Image Diffusion process—network architecture, denoising, training, sampling, conditional generation, guidance, and fine-tuning.
• Adapt pre-trained Stable Diffusion models with specific datasets using Low-Raw Adaptation (LoRA) for efficient fine-tuning.
• Develop skills to run open and closed image generation models and integrate them into other applications.
• Understand the fundamentals of Language Models (LLMs) and their specific applications within the AEC industry.
• Identify the capabilities of LLMs across classification, knowledge retrieval, Q&A, summarization, and image captioning.
• Compare and contrast different techniques for utilising LLMs, such as prompt engineering, zero-shot and few-shot learning and augmented retrieval.
• Create custom datasets tailored to specific architectural and design needs, and understand the importance of knowledge databases for Retrieval-Augmented Generation (RAG).
• Evaluate the advantages and disadvantages of running LLMs locally versus using cloud-based APIs.
• Develop practical skills in setting up LLMs from scratch and utilising open-source tools for experimentation and prototyping.