RoboMosaic

MRAC Workshop 2.2

Introduction

We know that trash is a problem, specifically with respect to construction and demolition waste. The saying is supposed to go, ‘One man’s trash is another’s treasure.’ but at the moment we are missing the processes to intervene in the massive flows of construction and demolition wasted that are sent to landfills. So it can’t become another’s treasure, it is destined simply to be trash. In a sense it’s a problem of unconstrained entropy. The organization that is required to bring order to mounds of construction and demolition waste and convert it into treasure is lacking.

And not for invalid reasons. The heterogeneity of waste streams with a variety of materials and sizes means that sorting of trash into what is useful and what is not is actually a very complex task. The complexity of the task has until very recently necessitated human labour, which is both expensive and not particularly engaging or rewarding for the humans involved. With advances in technology however, robots are now sufficiently capable of the sensing, computing, and dexterity required to handle some of the complex tasks associated with the recycling and reuse of waste.

This was the context of our workshop. With a continuous aim to convert a very linear supply chain into a more cyclical one, we explored the potential of using a robot to scan, sort, and manipulate shards of ceramic tile, so that they could be reused in a mosaic.

Being in Barcelona, the work of Gaudi is an ever-present influence. Specific to this workshop, his use of trencadis was our inspiration. In this method, broken tiles are used to form intricate geometric shapes and patterns. Gaudi’s work often involved taking tiles left over from other construction projects and breaking them into smaller pieces, but the technique works well enough for recycled tiles that are already broken as well.

We attempted to emulate this technique through a workflow involving scanning, algorithmic design, and robotic pick and place.

Equipment

The equipment used in this project included a robot (UR10e), a depth camera (Intel RealSense D405), and an end effector with a suction cup to pick up and place the tiles, with ROS being used for control, and Open3D used for reconstruction. The design software used was Rhino/Grasshopper, with specific emphasis on the Kangaroo and OpenNest plugins. The materials used were broken tiles, mortar, and grout.

Workflow

Our worklfow involved scanning, designing, and finally robotic fabrication. Simply put: scan, design, place.

This workflow would complete one cycle of the ideal iterative process described in the image below. Further cycles would involve scanning the work in progress for quality control and to inform follow-on design decisions, but that was beyond the scope of this workshop.

Scanning and the Digital Library

First, we built a digital library of the mosaic materials. We placed the tiles on a surface of known dimensions and position relative to the robot. We scanned the shards using the RealSense camera which produce a point cloud and used Open3D for industrial reconstruction. This gave us a mesh for each shard.

At the same time, we used Open3D to clip the base plane (table) leaving only the shards. The script also extracted the center of the top face of each shard, and computed the normal vector, which would later be used for the picking plane.

We also scanned the backing tile on which the shards would be placed, so as to provide an accurate placement surface in our design step. This completed the data collection.

Design Intentions

We liked the idea of using a monochromatic colour scheme and of using the tile boundaries in contrast with dark grouting for our design.

We thought this could maybe used as a way of replacing signage in public spaces. Posters, signs, floor markings, and other means of communicating information about the use of a space are often added as an afterthought, and can detract from a sense of cohesion in a space. Therefore, we thought that by providing a means to provide information within the aesthetic designs of a space might be a useful avenue to explore for our project.

Initially we experimented with manually manipulated designs in grasshopper/Rhino, but later moved to algorithmic methods. Some of the tools we used were OpenNest and Kangaroo.

Common to both approaches was the extraction of shard edges, which would facitilitate their manipulation in the design. To do this we projected the meshes onto a base plane, extracted the contour, and then rebuilt the line until we obtained a useful representation of the shard edges. Especially whether they were straight or curved.

Our OpenNest approach was pretty rudimentary. Basically, we would define a shape within the boundaries of the placement tile, and then run OpenNest to pack as many shards into our defined shape as possible. The features in the panel were thus emphasized by the space between tiles, rather than by any sort of filtering or alignment of the shard edges, which was our ultimate goal. The results were satisfactory, but it left large areas uncovered by tile and needing to be grouted, which partly defeats the purpose of using as much recycled tile as possible.

The kangaroo approach was more refined. First, the edges of the shards were sorted according to their linearity and length. Then, those edges were aligned to the design curves. Finally, the remaining shards were packed as densely as possible into any open space. The results were promising, but unfortunately due to the time constraints of the workshop we weren’t able to explore it as fully as we would have wanted.

During the design process and before getting into the robotic workflow, we started with manual exploration. This helped us to develop a bit of intuition about the placement of tiles that we would be doing later with robots. It also allowed us to start fleshing out our design concepts very quickly.

Pick and Place

Common again to both processes was the generation of robotic path planning. The relative transformations of each shard from their initial position to their new position in the design was used to generate a set of pick and place instructions for the robot.

Finally came the pick and place with the robot. The TCP was modified to include a suction cup at the end of the robot. The robot head would approach to an intermediate point above the tile, move to the surface of the shard to pick it, then move the shard to its new position on the placement tile.

Results

We produced several mosaics to test our workflow.

Next Steps

There are several aspects of this workshop that we would be interested in expanding on the future.

Improving the design algorithms. At this point our design process was very hands on, and not adaptive. Ideally we would be able to set a single global design and have a design algorithm place recently scanned shards accordingly.

Quality control/cyclical scanning. At this point, once the shards are placed, there is no further data collection. Ideally, the mosaic could be scanned after each application of new shards to update the design algorithm and placement requirements. This would bring us inline with the ideal workflow, where the design rules are predetermined, and the scanning and placing happen concurrently while continually updating.

Finally, we were asked to imagine where this process could be applied. An application that is very interesting to us it the mosaic tiling of otherwise utilitarian and mundane public spaces, such as walkways, paths, and parking lots. These areas generally have a great deal of signage pertaining to their use, and also are so large that they do not benefit from an artisanal approach, and are instead paved since it is efficient and less expensive to do so.

For instance, when walking beside the Palau Nacional, we noticed that there is a large and uninspiring asphalt parking lot directly beside it. Juxtaposed against all of the care and craftsmanship that went into making a monumental building like the Palau Nacional, that parking lot felt out of place. Perhaps places like these can be adapted to the spirit of their surroundings with the help of robotic craftsmanship. Our hope is that this would make our public spaces more inviting and human, while maintaining the durability and utility that is required of them.