Publications | Computational Design Group

2024

WasteBanned: Supporting Zero Waste Fashion Design Through Linked Edits

Ruowang Zhang, Stefanie Mueller, Gilbert Louis Bernstein, Adriana Schulz, and Mackenzie Leake

In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, Pittsburgh, PA, USA, 2024

Abs DOI Bib

The commonly used cut-and-sew garment construction process, in which 2D fabric panels are cut from sheets of fabric and assembled into 3D garments, contributes to widespread textile waste in the fashion industry. There is often a significant divide between the design of the garment and the layout of the panels. One opportunity for bridging this gap is the emerging study and practice of zero waste fashion design, which involves creating clothing designs with maximum layout efficiency. Enforcing the strict constraints of zero waste sewing is challenging, as edits to one region of the garment necessarily affect neighboring panels. Based on our formative work to understand this emerging area within fashion design, we present WasteBanned, a tool that combines CAM and CAD to help users prioritize efficient material usage, work within these zero waste constraints, and edit existing zero waste garment patterns. Our user evaluation indicates that our tool helps fashion designers edit zero waste patterns to fit different bodies and add stylistic variation, while creating highly efficient fabric layouts.
@inproceedings{zhang2024wastebanned, author = {Zhang, Ruowang and Mueller, Stefanie and Bernstein, Gilbert Louis and Schulz, Adriana and Leake, Mackenzie}, title = {WasteBanned: Supporting Zero Waste Fashion Design Through Linked Edits}, year = {2024}, isbn = {9798400706288}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3654777.3676395}, doi = {10.1145/3654777.3676395}, booktitle = {Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology}, articleno = {110}, numpages = {13}, keywords = {Design tools, Fabrication, Sustainable design, Zero waste}, location = {Pittsburgh, PA, USA}, series = {UIST '24}, }
Computational Illusion Knitting

Amy Zhu, Yuxuan Mei, Benjamin Jones, Zachary Tatlock, and Adriana Schulz

ACM Trans. Graph., Jul 2024

Abs DOI Bib

Illusion-knit fabrics reveal distinct patterns or images depending on the viewing angle. Artists have manually achieved this effect by exploiting "microgeometry," i.e., small differences in stitch heights. However, past work in computational 3D knitting does not model or exploit designs based on stitch height variation. This paper establishes a foundation for exploring illusion knitting in the context of computational design and fabrication. We observe that the design space is highly constrained, elucidate these constraints, and derive strategies for developing effective, machine-knittable illusion patterns. We partially automate these strategies in a new interactive design tool that reduces difficult patterning tasks to familiar image editing tasks. Illusion patterns also uncover new fabrication challenges regarding mixed colorwork and texture; we describe new algorithms for mitigating fabrication failures and ensuring high-quality knit results.
@article{zhu2024illustionknitting, author = {Zhu, Amy and Mei, Yuxuan and Jones, Benjamin and Tatlock, Zachary and Schulz, Adriana}, title = {Computational Illusion Knitting}, year = {2024}, issue_date = {July 2024}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {43}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/3658231}, doi = {10.1145/3658231}, journal = {ACM Trans. Graph.}, month = jul, articleno = {152}, numpages = {13}, keywords = {illusion knitting, machine knitting, knitting, fabrication}, }
FabHacks: Transform Everyday Objects into Home Hacks Leveraging a Solver-aided DSL

Yuxuan Mei, Benjamin Jones, Dan Cascaval, Jennifer Mankoff, Etienne Vouga, and Adriana Schulz

In Proceedings of the 9th ACM Symposium on Computational Fabrication, Aarhus, Denmark, Jul 2024

Abs DOI Bib

Storage, organizing, and decorating are important aspects of home design. Buying commercial items for many of these tasks, this can be costly, and reuse is more sustainable. An alternative is a “home hack,” i.e., a functional assembly constructed from existing household items. However, coming up with such hacks requires combining objects to make a physically valid design, which might be difficult to test if they are large, require nailing or screwing to the wall, or if the designer has mobility limitations. We present a design and visualization system, FabHacks, for creating workable functional assemblies. The system is based on a new solver-aided domain-specific language (S-DSL) called FabHaL. By analyzing existing home hacks shared online, we create a design abstraction for connecting household items using predefined connection types. We also provide a UI for designing hack assemblies that fulfill a given specification. FabHacks leverages a physics-based solver that finds the expected physical configuration of an assembly design. Our validation includes a user study with our UI, which shows that users can easily create assemblies and explore a range of designs.
@inproceedings{mei2024fabhacks, author = {Mei, Yuxuan and Jones, Benjamin and Cascaval, Dan and Mankoff, Jennifer and Vouga, Etienne and Schulz, Adriana}, title = {FabHacks: Transform Everyday Objects into Home Hacks Leveraging a Solver-aided DSL}, year = {2024}, isbn = {9798400704963}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3639473.3665788}, doi = {10.1145/3639473.3665788}, booktitle = {Proceedings of the 9th ACM Symposium on Computational Fabrication}, articleno = {4}, numpages = {16}, keywords = {domain-specific languages, fabrication, sustainability}, location = {Aarhus, Denmark}, series = {SCF '24}, }
Understanding and Supporting Debugging Workflows in CAD

Felix Hähnlein, Gilbert Bernstein, and Adriana Schulz

In Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology, Pittsburgh, PA, USA, Jul 2024

Abs DOI Bib

One of the core promises of parametric Computer-Aided Design (CAD) is that users can easily edit their model at any point in time. However, due to the ambiguity of changing references to intermediate, updated geometry, parametric edits can lead to reference errors which are difficult to fix in practice. We claim that debugging reference errors remains challenging because CAD systems do not provide users with tools to understand where the error happened and how to fix it. To address these challenges, we prototype a graphical debugging tool, DeCAD, which helps comparing CAD model states both across operations and across edits. In a qualitative lab study, we use DeCAD as a probe to understand specific challenges that users face and what workflows they employ to overcome them. We conclude with design implications for future debugging tool developers.
@inproceedings{hahnlein2024debugging, author = {H\"{a}hnlein, Felix and Bernstein, Gilbert and Schulz, Adriana}, title = {Understanding and Supporting Debugging Workflows in CAD}, year = {2024}, isbn = {9798400706288}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3654777.3676353}, doi = {10.1145/3654777.3676353}, booktitle = {Proceedings of the 37th Annual ACM Symposium on User Interface Software and Technology}, articleno = {124}, numpages = {14}, location = {Pittsburgh, PA, USA}, series = {UIST '24}, }
Dense Server Design for Immersion Cooling

Milin Kodnongbua, Zachary Englhardt, Ricardo Bianchini, Rodrigo Fonseca, Alvin Lebeck, Daniel S. Berger, Vikram Iyer, Fiodar Kazhamiaka, and Adriana Schulz

ACM Trans. Graph., Nov 2024

Abs DOI Bib

The growing demands for computational power in cloud computing have led to a significant increase in the deployment of high-performance servers. The growing power consumption of servers and the heat they produce is on track to outpace the capacity of conventional air cooling systems, necessitating more efficient cooling solutions such as liquid immersion cooling. The superior heat exchange capabilities of immersion cooling both eliminates the need for bulky heat sinks, fans, and air flow channels while also unlocking the potential go beyond conventional 2D blade servers to three-dimensional designs. In this work, we present a computational framework to explore designs of servers in three-dimensional space, specifically targeting the maximization of server density within immersion cooling tanks. Our tool is designed to handle a variety of physical and electrical server design constraints. We demonstrate our optimized designs can reduce server volume by 25–52% compared to traditional flat server designs. This increased density reduces land usage as well as the amount of liquid used for immersion, with significant reduction in the carbon emissions embodied in datacenter buildings. We further create physical prototypes to simulate dense server designs and perform real-world experiments in an immersion cooling tank demonstrating they operate at safe temperatures. This approach marks a critical step forward in sustainable and efficient datacenter management.
@article{kodnongbua2024servers, author = {Kodnongbua, Milin and Englhardt, Zachary and Bianchini, Ricardo and Fonseca, Rodrigo and Lebeck, Alvin and Berger, Daniel S. and Iyer, Vikram and Kazhamiaka, Fiodar and Schulz, Adriana}, title = {Dense Server Design for Immersion Cooling}, year = {2024}, issue_date = {December 2024}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {43}, number = {6}, issn = {0730-0301}, url = {https://doi.org/10.1145/3687965}, doi = {10.1145/3687965}, journal = {ACM Trans. Graph.}, month = nov, articleno = {210}, numpages = {20}, keywords = {server design, fabrication-oriented design, immersion cooling, sustainable design}, }

Zero-shot Sequential Neuro-symbolic Reasoning for Automatically Generating Architecture Schematic Designs

Milin Kodnongbua, Lawrence H. Curtis, and Adriana Schulz

Nov 2024

@misc{kodnongbua2024gptarch,
  title = {Zero-shot Sequential Neuro-symbolic Reasoning for Automatically Generating Architecture Schematic Designs},
  author = {Kodnongbua, Milin and Curtis, Lawrence H. and Schulz, Adriana},
  year = {2024},
  eprint = {2402.00052},
  archiveprefix = {arXiv},
  primaryclass = {cs.AI},
  url = {https://arxiv.org/abs/2402.00052},
}

DeltaLCA: Comparative Life-Cycle Assessment for Electronics Design

Zhihan Zhang, Felix Hähnlein, Yuxuan Mei, Zachary Englhardt, Shwetak Patel, Adriana Schulz, and Vikram Iyer

Proc. ACM Interact. Mob. Wearable Ubiquitous Technol., Mar 2024

Abs DOI Bib

Reducing the environmental footprint of electronics and computing devices requires new tools that empower designers to make informed decisions about sustainability during the design process itself. This is not possible with current tools for life cycle assessment (LCA) which require substantial domain expertise and time to evaluate the numerous chips and other components that make up a device. We observe first that informed decision-making does not require absolute metrics and can instead be done by comparing designs. Second, we can use domain-specific heuristics to perform these comparisons. We combine these insights to develop DeltaLCA, an open-source interactive design tool that addresses the dual challenges of automating life cycle inventory generation and data availability by performing comparative analyses of electronics designs. Users can upload standard design files from Electronic Design Automation (EDA) software and the tool will guide them through determining which one has greater carbon footprints. DeltaLCA leverages electronics-specific LCA datasets and heuristics and tries to automatically rank the two designs, prompting users to provide additional information only when necessary. We show through case studies DeltaLCA achieves the same result as evaluating full LCAs, and that it accelerates LCA comparisons from eight expert-hours to a single click for devices with 30 components, and 15 minutes for more complex devices with 100 components.
@article{zhang2023deltalca, author = {Zhang, Zhihan and H\"{a}hnlein, Felix and Mei, Yuxuan and Englhardt, Zachary and Patel, Shwetak and Schulz, Adriana and Iyer, Vikram}, title = {DeltaLCA: Comparative Life-Cycle Assessment for Electronics Design}, year = {2024}, issue_date = {March 2024}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {8}, number = {1}, url = {https://doi.org/10.1145/3643561}, doi = {10.1145/3643561}, journal = {Proc. ACM Interact. Mob. Wearable Ubiquitous Technol.}, month = mar, articleno = {29}, numpages = {29}, keywords = {Domain-Specific Heuristics, Life Cycle Assessment, Linear Programming, Sustainable Computing}, }

2023

Toward Sub-Gram Helicopters: Designing a Miniaturized Flybar for Passive Stability

Kyle Johnson, Vicente Arroyos, Raul Villanueva, Adriana Schulz, Sawyer Fuller, and Vikram Iyer

In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Mar 2023

DOI Bib

@inproceedings{johnson2023flybar,
  author = {Johnson, Kyle and Arroyos, Vicente and Villanueva, Raul and Schulz, Adriana and Fuller, Sawyer and Iyer, Vikram},
  booktitle = {2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  title = {Toward Sub-Gram Helicopters: Designing a Miniaturized Flybar for Passive Stability},
  year = {2023},
  volume = {},
  number = {},
  pages = {2701-2708},
  keywords = {Damping;Low voltage;Three-dimensional displays;Rotors;Tail;Reliability engineering;Laminates},
  doi = {10.1109/IROS55552.2023.10342256},
}

How Can Large Language Models Help Humans in Design and Manufacturing?

Liane Makatura, Michael Foshey, Bohan Wang, Felix HähnLein, Pingchuan Ma, Bolei Deng, Megan Tjandrasuwita, Andrew Spielberg, Crystal Elaine Owens, Peter Yichen Chen, Allan Zhao, Amy Zhu, Wil J Norton, Edward Gu, Joshua Jacob, Yifei Li, Adriana Schulz, and Wojciech Matusik

Mar 2023

arXiv Bib

@misc{makatura2023largelanguagemodelshelp,
  title = {How Can Large Language Models Help Humans in Design and Manufacturing?},
  author = {Makatura, Liane and Foshey, Michael and Wang, Bohan and HähnLein, Felix and Ma, Pingchuan and Deng, Bolei and Tjandrasuwita, Megan and Spielberg, Andrew and Owens, Crystal Elaine and Chen, Peter Yichen and Zhao, Allan and Zhu, Amy and Norton, Wil J and Gu, Edward and Jacob, Joshua and Li, Yifei and Schulz, Adriana and Matusik, Wojciech},
  year = {2023},
  eprint = {2307.14377},
  archiveprefix = {arXiv},
  primaryclass = {cs.CL},
  url = {https://arxiv.org/abs/2307.14377},
}

A Lineage-Based Referencing DSL for Computer-Aided Design

Dan Cascaval, Rastislav Bodik, and Adriana Schulz

Proc. ACM Program. Lang., Jun 2023

Abs DOI Bib

3D Computer-Aided Design (CAD) modeling is ubiquitous in mechanical engineering and design. Modern CAD models are programs that produce geometry and can be used to implement high-level geometric changes by modifying input parameters. While there has been a surge of recent interest in program-based tooling for the CAD domain, one fundamental problem remains unsolved. CAD programs pass geometric arguments to operations using references, which are queries that select elements from the constructed geometry according to programmer intent. The challenge is designing reference semantics that can express programmer intent across all geometric topologies achievable with model parameters, including topologies where the desired elements are not present. In current systems, both users and automated tools may create invalid models when parameters are changed, as references to geometric elements are lost or silently and arbitrarily switched. While existing CAD systems use heuristics to attempt to infer user intent in cases of this undefined behavior, this best-effort solution is not suitable for constructing automated tools to edit and optimize CAD programs. We analyze the failure modes of existing referencing schemes and formalize a set of criteria on which to evaluate solutions to the CAD referencing problem. In turn, we propose a domain-specific language that exposes references as a first-class language construct, using user-authored queries to introspect element history and define references safely over all paths. We give a semantics for fine-grained element lineage that can subsequently be queried; and show that our language meets the desired properties. Finally, we provide an implementation of a lineage-based referencing system in a 2.5D CAD kernel, demonstrating realistic referencing scenarios and illustrating how our system safely represents models that cause reference breakage in existing CAD systems.
@article{cascaval2023lineage, author = {Cascaval, Dan and Bodik, Rastislav and Schulz, Adriana}, title = {A Lineage-Based Referencing DSL for Computer-Aided Design}, year = {2023}, issue_date = {June 2023}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {7}, number = {PLDI}, url = {https://doi.org/10.1145/3591223}, doi = {10.1145/3591223}, journal = {Proc. ACM Program. Lang.}, month = jun, articleno = {109}, numpages = {24}, keywords = {shape modeling, persistent naming, computer-aided design programs}, }

Self-Supervised Representation Learning for CAD

Benjamin T. Jones, Michael Hu, Milin Kodnongbua, Vladimir G. Kim, and Adriana Schulz

In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Jun 2023

DOI Bib

@inproceedings{jones2023self,
  author = {Jones, Benjamin T. and Hu, Michael and Kodnongbua, Milin and Kim, Vladimir G. and Schulz, Adriana},
  booktitle = {2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  title = {Self-Supervised Representation Learning for CAD},
  year = {2023},
  volume = {},
  number = {},
  pages = {21327-21336},
  keywords = {Geometry;Training;Representation learning;Design automation;Supervised learning;Reverse engineering;Self-supervised learning;Self-supervised or unsupervised representation learning},
  doi = {10.1109/CVPR52729.2023.02043},
}

B-rep Matching for Collaborating Across CAD Systems

Benjamin Jones, James Noeckel, Milin Kodnongbua, Ilya Baran, and Adriana Schulz

ACM Trans. Graph., Jul 2023

Abs DOI Bib

Large Computer-Aided Design (CAD) projects usually require collaboration across many different CAD systems as well as applications that interoperate with them for manufacturing, visualization, or simulation. A fundamental barrier to such collaborations is the ability to refer to parts of the geometry (such as a specific face) robustly under geometric and/or topological changes to the model. Persistent referencing schemes are a fundamental aspect of most CAD tools, but models that are shared across systems cannot generally make use of these internal referencing mechanisms, creating a challenge for collaboration. In this work, we address this issue by developing a novel learning-based algorithm that can automatically find correspondences between two CAD models using the standard representation used for sharing models across CAD systems: the Boundary-Representation (B-rep). Because our method works directly on B-reps it can be generalized across different CAD applications enabling collaboration.
@article{jones2023brepmatching, author = {Jones, Benjamin and Noeckel, James and Kodnongbua, Milin and Baran, Ilya and Schulz, Adriana}, title = {B-rep Matching for Collaborating Across CAD Systems}, year = {2023}, issue_date = {August 2023}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {42}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/3592125}, doi = {10.1145/3592125}, journal = {ACM Trans. Graph.}, month = jul, articleno = {104}, numpages = {13}, keywords = {computer-aided design, parametric modeling, geometric correspondence, machine learning}, }
ReparamCAD: Zero-shot CAD Re-Parameterization for Interactive Manipulation

Milin Kodnongbua, Benjamin Jones, Maaz Bin Safeer Ahmad, Vladimir Kim, and Adriana Schulz

In SIGGRAPH Asia 2023 Conference Papers, Sydney, NSW, Australia, Jul 2023

Abs DOI Bib

Parametric CAD models encode entire families of shapes that should, in principle, be easy for designers to explore. However, in practice, parametric CAD models can be difficult to manipulate due to implicit semantic constraints among parameter values. Finding and enforcing these semantic constraints solely from geometry or programmatic shape representations is not possible because these constraints ultimately reflect design intent. They are informed by the designer’s experience and semantics in the real world. To address this challenge, we introduce ReparamCAD, a zero-shot pipeline that leverages pre-trained large language and image model to infer meaningful space of variations for a shape We then re-parameterize a new constrained parametric CAD program that captures these variations, enabling effortless exploration of the design space along meaningful design axes. We evaluated our approach through five examples and a user study. The result showed that the inferred spaces are meaningful and comparable to those defined by experts. Code and data are at: https://github.com/milmillin/ReparamCAD.
@inproceedings{kodnongbua2023reparamcad, author = {Kodnongbua, Milin and Jones, Benjamin and Ahmad, Maaz Bin Safeer and Kim, Vladimir and Schulz, Adriana}, title = {ReparamCAD: Zero-shot CAD Re-Parameterization for Interactive Manipulation}, year = {2023}, isbn = {9798400703157}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, url = {https://doi.org/10.1145/3610548.3618219}, doi = {10.1145/3610548.3618219}, booktitle = {SIGGRAPH Asia 2023 Conference Papers}, articleno = {69}, numpages = {12}, keywords = {parametric modeling, program synthesis}, location = {Sydney, NSW, Australia}, series = {SA '23}, }
Generative Design of Sheet Metal Structures

Amir Barda, Guy Tevet, Adriana Schulz, and Amit Haim Bermano

ACM Trans. Graph., Jul 2023

Abs DOI Bib

Sheet Metal (SM) fabrication is perhaps one of the most common metalworking technique.Despite its prevalence, SM design is manual and costly, with rigorous practices that restrict the search space, yielding suboptimal results.In contrast, we present a framework for the first automatic design of SM parts. Focusing on load bearing applications, our novel system generates a high-performing manufacturable SM that adheres to the numerous constraints that SM design entails:The resulting part minimizes manufacturing costs while adhering to structural, spatial, and manufacturing constraints. In other words, the part should be strong enough, not disturb the environment, and adhere to the manufacturing process. These desiderata sum up to an elaborate, sparse, and expensive search space.Our generative approach is a carefully designed exploration process, comprising two steps. In Segment Discovery connections from the input load to attachable regions are accumulated, and during Segment Composition the most performing valid combination is searched for.For Discovery, we define a slim grammar, and sample it for parts using a Markov-Chain Monte Carlo (MCMC) approach, ran in intercommunicating instances (i.e, chains) for diversity. This, followed by a short continuous optimization, enables building a diverse and high-quality library of substructures. During Composition, a valid and minimal cost combination of the curated substructures is selected. To improve compliance significantly without additional manufacturing costs, we reinforce candidate parts onto themselves — a unique SM capability called self-riveting. we provide our code and data in https://github.com/amir90/AutoSheetMetal.We show our generative approach produces viable parts for numerous scenarios. We compare our system against a human expert and observe improvements in both part quality and design time. We further analyze our pipeline’s steps with respect to resulting quality, and have fabricated some results for validation.We hope our system will stretch the field of SM design, replacing costly expert hours with minutes of standard CPU, making this cheap and reliable manufacturing method accessible to anyone.
@article{barda2023generative, author = {Barda, Amir and Tevet, Guy and Schulz, Adriana and Bermano, Amit Haim}, title = {Generative Design of Sheet Metal Structures}, year = {2023}, issue_date = {August 2023}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {42}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/3592444}, doi = {10.1145/3592444}, journal = {ACM Trans. Graph.}, month = jul, articleno = {116}, numpages = {13}, keywords = {computational fabrication, generative design, sheet metal}, }
Neurosymbolic Models for Computer Graphics

Daniel Ritchie, Paul Guerrero , R. Kenny Jones, Niloy J. Mitra, Adriana Schulz, Karl D. D. Willis, and Jiajun Wu

Computer Graphics Forum, Jul 2023

Abs DOI Bib

Abstract Procedural models (i.e. symbolic programs that output visual data) are a historically-popular method for representing graphics content: vegetation, buildings, textures, etc. They offer many advantages: interpretable design parameters, stochastic variations, high-quality outputs, compact representation, and more. But they also have some limitations, such as the difficulty of authoring a procedural model from scratch. More recently, AI-based methods, and especially neural networks, have become popular for creating graphic content. These techniques allow users to directly specify desired properties of the artifact they want to create (via examples, constraints, or objectives), while a search, optimization, or learning algorithm takes care of the details. However, this ease of use comes at a cost, as it’s often hard to interpret or manipulate these representations. In this state-of-the-art report, we summarize research on neurosymbolic models in computer graphics: methods that combine the strengths of both AI and symbolic programs to represent, generate, and manipulate visual data. We survey recent work applying these techniques to represent 2D shapes, 3D shapes, and materials & textures. Along the way, we situate each prior work in a unified design space for neurosymbolic models, which helps reveal underexplored areas and opportunities for future research.
@article{ritchie2023neurosymbolic, author = {Ritchie, Daniel and Guerrero, Paul and Jones, R. Kenny and Mitra, Niloy J. and Schulz, Adriana and Willis, Karl D. D. and Wu, Jiajun}, title = {Neurosymbolic Models for Computer Graphics}, journal = {Computer Graphics Forum}, volume = {42}, number = {2}, pages = {545-568}, keywords = {CCS Concepts, • Computing methodologies → Shape modeling; Reflectance modeling; Texturing; Neural networks; Computer vision, • Software and its engineering → Domain specific languages; Programming by example}, doi = {https://doi.org/10.1111/cgf.14775}, url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.14775}, eprint = {https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14775}, year = {2023}, }

Mates2Motion: Learning How Mechanical CAD Assemblies Work

James Noeckel, Benjamin T. Jones, Karl Willis, Brian Curless, and Adriana Schulz

Jul 2023

arXiv Bib

@misc{noeckel2023mates2motion,
  title = {Mates2Motion: Learning How Mechanical CAD Assemblies Work},
  author = {Noeckel, James and Jones, Benjamin T. and Willis, Karl and Curless, Brian and Schulz, Adriana},
  year = {2023},
  eprint = {2208.01779},
  archiveprefix = {arXiv},
  primaryclass = {cs.CV},
  url = {https://arxiv.org/abs/2208.01779},
}

2022

Computational design of passive grippers

Milin Kodnongbua, Ian Good, Yu Lou, Jeffrey Lipton, and Adriana Schulz

ACM Trans. Graph., Jul 2022

Abs DOI Bib

This work proposes a novel generative design tool for passive grippers—robot end effectors that have no additional actuation and instead leverage the existing degrees of freedom in a robotic arm to perform grasping tasks. Passive grippers are used because they offer interesting trade-offs between cost and capabilities. However, existing designs are limited in the types of shapes that can be grasped. This work proposes to use rapid-manufacturing and design optimization to expand the space of shapes that can be passively grasped. Our novel generative design algorithm takes in an object and its positioning with respect to a robotic arm and generates a 3D printable passive gripper that can stably pick the object up. To achieve this, we address the key challenge of jointly optimizing the shape and the insert trajectory to ensure a passively stable grasp. We evaluate our method on a testing suite of 22 objects (23 experiments), all of which were evaluated with physical experiments to bridge the virtual-to-real gap. Code and data are at https://homes.cs.washington.edu/ milink/passive-gripper/
@article{kodnongbua2022grippers, author = {Kodnongbua, Milin and Good, Ian and Lou, Yu and Lipton, Jeffrey and Schulz, Adriana}, title = {Computational design of passive grippers}, year = {2022}, issue_date = {July 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {41}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/3528223.3530162}, doi = {10.1145/3528223.3530162}, journal = {ACM Trans. Graph.}, month = jul, articleno = {149}, numpages = {11}, keywords = {additive manufacturing, fabrication, generative design, passive gripper}, }
Differentiable 3D CAD Programs for Bidirectional Editing

Dan Cascaval, Mira Shalah, Phillip Quinn, Ras Bodik, Maneesh Agrawala, and Adriana Schulz

Computer Graphics Forum, Jul 2022

Abs DOI Bib

Abstract Modern CAD tools represent 3D designs not only as geometry, but also as a program composed of geometric operations, each of which depends on a set of parameters. Program representations enable meaningful and controlled shape variations via parameter changes. However, achieving desired modifications solely through parameter editing is challenging when CAD models have not been explicitly authored to expose select degrees of freedom in advance. We introduce a novel bidirectional editing system for 3D CAD programs. In addition to editing the CAD program, users can directly manipulate 3D geometry and our system infers parameter updates to keep both representations in sync. We formulate inverse edits as a set of constrained optimization objectives, returning plausible updates to program parameters that both match user intent and maintain program validity. Our approach implements an automatically differentiable domain-specific language for CAD programs, providing derivatives for this optimization to be performed quickly on any expressed program. Our system enables rapid, interactive exploration of a constrained 3D design space by allowing users to manipulate the program and geometry interchangeably during design iteration. While our approach is not designed to optimize across changes in geometric topology, we show it is expressive and performant enough for users to produce a diverse set of design variants, even when the CAD program contains a relatively large number of parameters.
@article{cascaval2022bidirectional, author = {Cascaval, Dan and Shalah, Mira and Quinn, Phillip and Bodik, Ras and Agrawala, Maneesh and Schulz, Adriana}, title = {Differentiable 3D CAD Programs for Bidirectional Editing}, journal = {Computer Graphics Forum}, volume = {41}, number = {2}, pages = {309-323}, keywords = {CCS Concepts, • Computing methodologies → Shape modeling, Graphics systems and interfaces}, doi = {https://doi.org/10.1111/cgf.14476}, url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.14476}, eprint = {https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14476}, year = {2022}, }
Co-Optimization of Design and Fabrication Plans for Carpentry

Haisen Zhao, Max Willsey, Amy Zhu, Chandrakana Nandi, Zachary Tatlock, Justin Solomon, and Adriana Schulz

ACM Trans. Graph., Mar 2022

Abs DOI Bib

Past work on optimizing fabrication plans given a carpentry design can provide Pareto-optimal plans trading off between material waste, fabrication time, precision, and other considerations. However, when developing fabrication plans, experts rarely restrict to a single design, instead considering families of design variations, sometimes adjusting designs to simplify fabrication. Jointly exploring the design and fabrication plan spaces for each design is intractable using current techniques. We present a new approach to jointly optimize design and fabrication plans for carpentered objects. To make this bi-level optimization tractable, we adapt recent work from program synthesis based on equality graphs (e-graphs), which encode sets of equivalent programs. Our insight is that subproblems within our bi-level problem share significant substructures. By representing both designs and fabrication plans in a new bag of parts (BOP) e-graph, we amortize the cost of optimizing design components shared among multiple candidates. Even using BOP e-graphs, the optimization space grows quickly in practice. Hence, we also show how a feedback-guided search strategy dubbed Iterative Contraction and Expansion on E-graphs (ICEE) can keep the size of the e-graph manageable and direct the search towards promising candidates. We illustrate the advantages of our pipeline through examples from the carpentry domain.
@article{zhao2022carpentryCoOptimization, author = {Zhao, Haisen and Willsey, Max and Zhu, Amy and Nandi, Chandrakana and Tatlock, Zachary and Solomon, Justin and Schulz, Adriana}, title = {Co-Optimization of Design and Fabrication Plans for Carpentry}, year = {2022}, issue_date = {June 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {41}, number = {3}, issn = {0730-0301}, url = {https://doi.org/10.1145/3508499}, doi = {10.1145/3508499}, journal = {ACM Trans. Graph.}, month = mar, articleno = {32}, numpages = {13}, keywords = {programming languages, Fabrication}, }

2021

Fabrication-Aware Reverse Engineering for Carpentry

James Noeckel, Haisen Zhao, Brian Curless, and Adriana Schulz

Computer Graphics Forum, Mar 2021

Abs DOI Bib

Abstract We propose a novel method to generate fabrication blueprints from images of carpentered items. While 3D reconstruction from images is a well-studied problem, typical approaches produce representations that are ill-suited for computer-aided design and fabrication applications. Our key insight is that fabrication processes define and constrain the design space for carpentered objects, and can be leveraged to develop novel reconstruction methods. Our method makes use of domain-specific constraints to recover not just valid geometry, but a semantically valid assembly of parts, using a combination of image-based and geometric optimization techniques. We demonstrate our method on a variety of wooden objects and furniture, and show that we can automatically obtain designs that are both easy to edit and accurate recreations of the ground truth. We further illustrate how our method can be used to fabricate a physical replica of the captured object as well as a customized version, which can be produced by directly editing the reconstructed model in CAD software.
@article{noekel2021reverseCarpentry, author = {Noeckel, James and Zhao, Haisen and Curless, Brian and Schulz, Adriana}, title = {Fabrication-Aware Reverse Engineering for Carpentry}, journal = {Computer Graphics Forum}, volume = {40}, number = {5}, pages = {301-314}, doi = {https://doi.org/10.1111/cgf.14375}, url = {https://onlinelibrary.wiley.com/doi/abs/10.1111/cgf.14375}, eprint = {https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14375}, year = {2021}, }

Robotic Jigsaw: A Non-Holonomic Cutting Robot and Path Planning Algorithm

Haisen Zhao, Yash Talwekar, Wenqing Lan, Chetan Sharma, Daniela Rus, Adriana Schulz, and Jeffrey I Lipton

In 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Mar 2021

DOI Bib

@inproceedings{zhao2021roboticJigsaws,
  author = {Zhao, Haisen and Talwekar, Yash and Lan, Wenqing and Sharma, Chetan and Rus, Daniela and Schulz, Adriana and Lipton, Jeffrey I},
  booktitle = {2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  title = {Robotic Jigsaw: A Non-Holonomic Cutting Robot and Path Planning Algorithm},
  year = {2021},
  volume = {},
  number = {},
  pages = {5809-5816},
  keywords = {Drilling;Fabrication;Shape;Conferences;Fixtures;Production facilities;Hardware},
  doi = {10.1109/IROS51168.2021.9636066},
}

AutoMate: a dataset and learning approach for automatic mating of CAD assemblies

Benjamin Jones, Dalton Hildreth, Duowen Chen, Ilya Baran, Vladimir G. Kim, and Adriana Schulz

ACM Trans. Graph., Dec 2021

Abs DOI Bib

Assembly modeling is a core task of computer aided design (CAD), comprising around one third of the work in a CAD workflow. Optimizing this process therefore represents a huge opportunity in the design of a CAD system, but current research of assembly based modeling is not directly applicable to modern CAD systems because it eschews the dominant data structure of modern CAD: parametric boundary representations (BREPs). CAD assembly modeling defines assemblies as a system of pairwise constraints, called mates, between parts, which are defined relative to BREP topology rather than in world coordinates common to existing work. We propose SB-GCN, a representation learning scheme on BREPs that retains the topological structure of parts, and use these learned representations to predict CAD type mates. To train our system, we compiled the first large scale dataset of BREP CAD assemblies, which we are releasing along with benchmark mate prediction tasks. Finally, we demonstrate the compatibility of our model with an existing commercial CAD system by building a tool that assists users in mate creation by suggesting mate completions, with 72.2% accuracy.
@article{jones2021automate, author = {Jones, Benjamin and Hildreth, Dalton and Chen, Duowen and Baran, Ilya and Kim, Vladimir G. and Schulz, Adriana}, title = {AutoMate: a dataset and learning approach for automatic mating of CAD assemblies}, year = {2021}, issue_date = {December 2021}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {40}, number = {6}, issn = {0730-0301}, url = {https://doi.org/10.1145/3478513.3480562}, doi = {10.1145/3478513.3480562}, journal = {ACM Trans. Graph.}, month = dec, articleno = {227}, numpages = {18}, keywords = {assembly-based modeling, boundary representation, computer-aided design, representation learning}, }
Computational Design of Knit Templates

Benjamin Jones, Yuxuan Mei, Haisen Zhao, Taylor Gotfrid, Jennifer Mankoff, and Adriana Schulz

ACM Trans. Graph., Dec 2021

Abs DOI Bib

We present an interactive design system for knitting that allows users to create template patterns that can be fabricated using an industrial knitting machine. Our interactive design tool is novel in that it allows direct control of key knitting design axes we have identified in our formative study and does so consistently across the variations of an input parametric template geometry. This is achieved with two key technical advances. First, we present an interactive meshing tool that lets users build a coarse quadrilateral mesh that adheres to their knit design guidelines. This solution ensures consistency across the parameter space for further customization over shape variations and avoids helices, promoting knittability. Second, we lift and formalize low-level machine knitting constraints to the level of this coarse quad mesh. This enables us to not only guarantee hand- and machine-knittability, but also provides automatic design assistance through auto-completion and suggestions. We show the capabilities through a set of fabricated examples that illustrate the effectiveness of our approach in creating a wide variety of objects and interactively exploring the space of design variations.
@article{jones2022knitTemplates, author = {Jones, Benjamin and Mei, Yuxuan and Zhao, Haisen and Gotfrid, Taylor and Mankoff, Jennifer and Schulz, Adriana}, title = {Computational Design of Knit Templates}, year = {2021}, issue_date = {April 2022}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {41}, number = {2}, issn = {0730-0301}, url = {https://doi.org/10.1145/3488006}, doi = {10.1145/3488006}, journal = {ACM Trans. Graph.}, month = dec, articleno = {16}, numpages = {16}, keywords = {quad-meshing, Knitting}, }
Rewrite rule inference using equality saturation

Chandrakana Nandi, Max Willsey, Amy Zhu, Yisu Remy Wang, Brett Saiki, Adam Anderson, Adriana Schulz, Dan Grossman, and Zachary Tatlock

Proc. ACM Program. Lang., Oct 2021

Abs DOI Bib

Many compilers, synthesizers, and theorem provers rely on rewrite rules to simplify expressions or prove equivalences. Developing rewrite rules can be difficult: rules may be subtly incorrect, profitable rules are easy to miss, and rulesets must be rechecked or extended whenever semantics are tweaked. Large rulesets can also be challenging to apply: redundant rules slow down rule-based search and frustrate debugging. This paper explores how equality saturation, a promising technique that uses e-graphs to apply rewrite rules, can also be used to infer rewrite rules. E-graphs can compactly represent the exponentially large sets of enumerated terms and potential rewrite rules. We show that equality saturation efficiently shrinks both sets, leading to faster synthesis of smaller, more general rulesets. We prototyped these strategies in a tool dubbed Ruler. Compared to a similar tool built on CVC4, Ruler synthesizes 5.8\texttimes smaller rulesets 25\texttimes faster without compromising on proving power. In an end-to-end case study, we show Ruler-synthesized rules which perform as well as those crafted by domain experts, and addressed a longstanding issue in a popular open source tool.
@article{nandi:2021:ruler, author = {Nandi, Chandrakana and Willsey, Max and Zhu, Amy and Wang, Yisu Remy and Saiki, Brett and Anderson, Adam and Schulz, Adriana and Grossman, Dan and Tatlock, Zachary}, title = {Rewrite rule inference using equality saturation}, year = {2021}, issue_date = {October 2021}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {5}, number = {OOPSLA}, url = {https://doi.org/10.1145/3485496}, doi = {10.1145/3485496}, journal = {Proc. ACM Program. Lang.}, month = oct, articleno = {119}, numpages = {28}, keywords = {Equality Saturation, Program Synthesis, Rewrite Rules}, }
Pareto gamuts: exploring optimal designs across varying contexts

Liane Makatura, Minghao Guo, Adriana Schulz, Justin Solomon, and Wojciech Matusik

ACM Trans. Graph., Jul 2021

Abs DOI Bib

Manufactured parts are meticulously engineered to perform well with respect to several conflicting metrics, like weight, stress, and cost. The best achievable trade-offs reside on the Pareto front, which can be discovered via performance-driven optimization. The objectives that define this Pareto front often incorporate assumptions about the context in which a part will be used, including loading conditions, environmental influences, material properties, or regions that must be preserved to interface with a surrounding assembly. Existing multi-objective optimization tools are only equipped to study one context at a time, so engineers must run independent optimizations for each context of interest. However, engineered parts frequently appear in many contexts: wind turbines must perform well in many wind speeds, and a bracket might be optimized several times with its bolt-holes fixed in different locations on each run. In this paper, we formulate a framework for variable-context multi-objective optimization. We introduce the Pareto gamut, which captures Pareto fronts over a range of contexts. We develop a global/local optimization algorithm to discover the Pareto gamut directly, rather than discovering a single fixed-context "slice" at a time. To validate our method, we adapt existing multi-objective optimization benchmarks to contextual scenarios. We also demonstrate the practical utility of Pareto gamut exploration for several engineering design problems.
@article{makatura:2021:paretoGamuts, author = {Makatura, Liane and Guo, Minghao and Schulz, Adriana and Solomon, Justin and Matusik, Wojciech}, title = {Pareto gamuts: exploring optimal designs across varying contexts}, year = {2021}, issue_date = {August 2021}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {40}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/3450626.3459750}, doi = {10.1145/3450626.3459750}, journal = {ACM Trans. Graph.}, month = jul, articleno = {171}, numpages = {17}, keywords = {computational design, computer-aided design, multi-objective optimization}, }

2020

VDAC: volume decompose-and-carve for subtractive manufacturing

Ali Mahdavi-Amiri, Fenggen Yu, Haisen Zhao, Adriana Schulz , and Hao Zhang

ACM Trans. Graph., Nov 2020

Abs DOI Bib

We introduce carvable volume decomposition for efficient 3-axis CNC machining of 3D freeform objects, where our goal is to develop a fully automatic method to jointly optimize setup and path planning. We formulate our joint optimization as a volume decomposition problem which prioritizes minimizing the number of setup directions while striving for a minimum number of continuously carvable volumes, where a 3D volume is continuously carvable, or simply carvable, if it can be carved with the machine cutter traversing a single continuous path. Geometrically, carvability combines visibility and monotonicity and presents a new shape property which had not been studied before. Given a target 3D shape and the initial material block, our algorithm first finds the minimum number of carving directions by solving a set cover problem. Specifically, we analyze cutter accessibility and select the carving directions based on an assessment of how likely they would lead to a small carvable volume decomposition. Next, to obtain a minimum decomposition based on the selected carving directions efficiently, we narrow down the solution search by focusing on a special kind of points in the residual volume, single access or SA points, which are points that can be accessed from one and only one of the selected carving directions. Candidate carvable volumes are grown starting from the SA points. Finally, we devise an energy term to evaluate the carvable volumes and their combinations, leading to the final decomposition. We demonstrate the performance of our decomposition algorithm on a variety of 2D and 3D examples and evaluate it against the ground truth, where possible, and solutions provided by human experts. Physically machined models are produced where each carvable volume is continuously carved following a connected Fermat spiral toolpath.
@article{amiri:2020:vdac, author = {Mahdavi-Amiri, Ali and Yu, Fenggen and Zhao, Haisen and Schulz, Adriana and Zhang, Hao}, title = {VDAC: volume decompose-and-carve for subtractive manufacturing}, year = {2020}, issue_date = {December 2020}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {39}, number = {6}, issn = {0730-0301}, url = {https://doi.org/10.1145/3414685.3417772}, doi = {10.1145/3414685.3417772}, journal = {ACM Trans. Graph.}, month = nov, articleno = {203}, numpages = {15}, keywords = {volume decomposition, toolpath planning, subtractive manufacturing, setup planning, rough-stage CNC machining}, }

2019

Carpentry compiler

Chenming Wu, Haisen Zhao, Chandrakana Nandi, Jeffrey I. Lipton, Zachary Tatlock, and Adriana Schulz

ACM Trans. Graph., Nov 2019

Abs DOI Bib

Traditional manufacturing workflows strongly decouple design and fabrication phases. As a result, fabrication-related objectives such as manufacturing time and precision are difficult to optimize in the design space, and vice versa. This paper presents HL-HELM, a high-level, domain-specific language for expressing abstract, parametric fabrication plans; it also introduces LL-HELM, a low-level language for expressing concrete fabrication plans that take into account the physical constraints of available manufacturing processes. We present a new compiler that supports the real-time, unoptimized translation of high-level, geometric fabrication operations into concrete, tool-specific fabrication instructions; this gives users immediate feedback on the physical feasibility of plans as they design them. HELM offers novel optimizations to improve accuracy and reduce fabrication time as well as material costs. Finally, optimized low-level plans can be interpreted as step-by-step instructions for users to actually fabricate a physical product. We provide a variety of example fabrication plans in the carpentry domain that are designed using our high-level language, show how the compiler translates and optimizes these plans to generate concrete low-level instructions, and present the final physical products fabricated in wood.
@article{wu:2019:carpentrycompiler, author = {Wu, Chenming and Zhao, Haisen and Nandi, Chandrakana and Lipton, Jeffrey I. and Tatlock, Zachary and Schulz, Adriana}, title = {Carpentry compiler}, year = {2019}, issue_date = {December 2019}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {38}, number = {6}, issn = {0730-0301}, url = {https://doi.org/10.1145/3355089.3356518}, doi = {10.1145/3355089.3356518}, journal = {ACM Trans. Graph.}, month = nov, articleno = {195}, numpages = {14}, keywords = {CAD, design for manufacturing, hardware abstractions, programming languages}, }
Learning to fly: computational controller design for hybrid UAVs with reinforcement learning

Jie Xu, Tao Du, Michael Foshey, Beichen Li , Bo Zhu, Adriana Schulz, and Wojciech Matusik

ACM Trans. Graph., Jul 2019

Abs DOI Bib

Hybrid unmanned aerial vehicles (UAV) combine advantages of multicopters and fixed-wing planes: vertical take-off, landing, and low energy use. However, hybrid UAVs are rarely used because controller design is challenging due to its complex, mixed dynamics. In this paper, we propose a method to automate this design process by training a mode-free, model-agnostic neural network controller for hybrid UAVs. We present a neural network controller design with a novel error convolution input trained by reinforcement learning. Our controller exhibits two key features: First, it does not distinguish among flying modes, and the same controller structure can be used for copters with various dynamics. Second, our controller works for real models without any additional parameter tuning process, closing the gap between virtual simulation and real fabrication. We demonstrate the efficacy of the proposed controller both in simulation and in our custom-built hybrid UAVs (Figure 1, 8). The experiments show that the controller is robust to exploit the complex dynamics when both rotors and wings are active in flight tests.
@article{xu:2019:LearningToFly, author = {Xu, Jie and Du, Tao and Foshey, Michael and Li, Beichen and Zhu, Bo and Schulz, Adriana and Matusik, Wojciech}, title = {Learning to fly: computational controller design for hybrid UAVs with reinforcement learning}, year = {2019}, issue_date = {August 2019}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {38}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/3306346.3322940}, doi = {10.1145/3306346.3322940}, journal = {ACM Trans. Graph.}, month = jul, articleno = {42}, numpages = {12}, keywords = {neural network controllers, hybrid UAVs}, }

2018

Robot Assisted Carpentry for Mass Customization

Jeffrey I Lipton, Adriana Schulz, Andrew Spielberg, Luis Trueba, Wojciech Matusik, and Daniela Rus

In 2018 IEEE International Conference on Robotics and Automation (ICRA), Jul 2018

DOI Bib

@inproceedings{lipton:2018:autosaw1,
  author = {Lipton, Jeffrey I and Schulz, Adriana and Spielberg, Andrew and Trueba, Luis and Matusik, Wojciech and Rus, Daniela},
  booktitle = {2018 IEEE International Conference on Robotics and Automation (ICRA)},
  title = {Robot Assisted Carpentry for Mass Customization},
  year = {2018},
  volume = {},
  number = {},
  pages = {3540-3547},
  keywords = {Fabrication;Robots;Solid modeling;Tools;Standards;Face;Connectors},
  doi = {10.1109/ICRA.2018.8460736},
}

InverseCSG: automatic conversion of 3D models to CSG trees

Tao Du, Jeevana Priya Inala, Yewen Pu, Andrew Spielberg, Adriana Schulz, Daniela Rus, Armando Solar-Lezama, and Wojciech Matusik

ACM Trans. Graph., Dec 2018

Abs DOI Bib

While computer-aided design is a major part of many modern manufacturing pipelines, the design files typically generated describe raw geometry. Lost in this representation is the procedure by which these designs were generated. In this paper, we present a method for reverse-engineering the process by which 3D models may have been generated, in the language of constructive solid geometry (CSG). Observing that CSG is a formal grammar, we formulate this inverse CSG problem as a program synthesis problem. Our solution is an algorithm that couples geometric processing with state-of-the-art program synthesis techniques. In this scheme, geometric processing is used to convert the mixed discrete and continuous domain of CSG trees to a pure discrete domain where modern program synthesizers excel. We demonstrate the efficiency and scalability of our algorithm on several different examples, including those with over 100 primitive parts. We show that our algorithm is able to find simple programs which are close to the ground truth, and demonstrate our method’s applicability in mesh re-editing. Finally, we compare our method to prior state-of-the-art. We demonstrate that our algorithm dominates previous methods in terms of resulting CSG compactness and runtime, and can handle far more complex input meshes than any previous method.
@article{Du:2018:InverseCSG, author = {Du, Tao and Inala, Jeevana Priya and Pu, Yewen and Spielberg, Andrew and Schulz, Adriana and Rus, Daniela and Solar-Lezama, Armando and Matusik, Wojciech}, title = {InverseCSG: automatic conversion of 3D models to CSG trees}, year = {2018}, issue_date = {December 2018}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {37}, number = {6}, issn = {0730-0301}, url = {https://doi.org/10.1145/3272127.3275006}, doi = {10.1145/3272127.3275006}, journal = {ACM Trans. Graph.}, month = dec, articleno = {213}, numpages = {16}, keywords = {CAD, CSG, procedural modeling}, }
Interactive exploration of design trade-offs

Adriana Schulz, Harrison Wang, Eitan Grinspun, Justin Solomon, and Wojciech Matusik

ACM Trans. Graph., Jul 2018

Abs DOI Bib

Typical design for manufacturing applications requires simultaneous optimization of conflicting performance objectives: Design variations that improve one performance metric may decrease another performance metric. In these scenarios, there is no unique optimal design but rather a set of designs that are optimal for different trade-offs (called Pareto-optimal). In this work, we propose a novel approach to discover the Pareto front, allowing designers to navigate the landscape of compromises efficiently. Our approach is based on a first-order approximation of the Pareto front, which allows entire neighborhoods rather than individual points on the Pareto front to be captured. In addition to allowing for efficient discovery of the Pareto front and the corresponding mapping to the design space, this approach allows us to represent the entire trade-off manifold as a small collection of patches that comprise a high-quality and piecewise-smooth approximation. We illustrate how this technique can be used for navigating performance trade-offs in computer-aided design (CAD) models.
@article{schulz:2018:Pareto, author = {Schulz, Adriana and Wang, Harrison and Grinspun, Eitan and Solomon, Justin and Matusik, Wojciech}, title = {Interactive exploration of design trade-offs}, year = {2018}, issue_date = {August 2018}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {37}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/3197517.3201385}, doi = {10.1145/3197517.3201385}, journal = {ACM Trans. Graph.}, month = jul, articleno = {131}, numpages = {14}, keywords = {shape modeling, pareto optimality, design for manufacturing}, }

2017

Interactive robogami: An end-to-end system for design of robots with ground locomotion

Adriana Schulz, Cynthia Sung, Andrew Spielberg, Wei Zhao, Robin Cheng, Eitan Grinspun, Daniela Rus, and Wojciech Matusik

The International Journal of Robotics Research, Jul 2017

Abs DOI Bib

This paper aims to democratize the design and fabrication of robots, enabling people of all skill levels to make robots without needing expert domain knowledge. Existing work in computational design and rapid fabrication has explored this question of customization for physical objects but so far has not been able to conquer the complexity of robot designs. We have developed Interactive Robogami, a tool for composition-based design of ground robots that can be fabricated as flat sheets and then folded into 3D structures. This rapid prototyping process enables users to create lightweight, affordable, and materially versatile robots with short turnaround time. Using Interactive Robogami, designers can compose new robot designs from a database of print-and-fold parts. The designs are tested for the users’ functional specifications via simulation and fabricated on user satisfaction. We present six robots designed and fabricated using a 3D printing based approach, as well as a larger robot cut from sheet metal. We have also conducted a user study that demonstrates that our tool is intuitive for novice designers and expressive enough to create a wide variety of ground robot designs.
@article{schulz:2017:robogami, author = {Schulz, Adriana and Sung, Cynthia and Spielberg, Andrew and Zhao, Wei and Cheng, Robin and Grinspun, Eitan and Rus, Daniela and Matusik, Wojciech}, title = {Interactive robogami: An end-to-end system for design of robots with ground locomotion}, journal = {The International Journal of Robotics Research}, volume = {36}, number = {10}, pages = {1131-1147}, year = {2017}, doi = {10.1177/0278364917723465}, url = {https://doi.org/10.1177/0278364917723465}, eprint = {https://doi.org/10.1177/0278364917723465}, }
Interactive design space exploration and optimization for CAD models

Adriana Schulz, Jie Xu , Bo Zhu, Changxi Zheng, Eitan Grinspun, and Wojciech Matusik

ACM Trans. Graph., Jul 2017

Abs DOI Bib

Computer Aided Design (CAD) is a multi-billion dollar industry used by almost every mechanical engineer in the world to create practically every existing manufactured shape. CAD models are not only widely available but also extremely useful in the growing field of fabrication-oriented design because they are parametric by construction and capture the engineer’s design intent, including manufacturability. Harnessing this data, however, is challenging, because generating the geometry for a given parameter value requires time-consuming computations. Furthermore, the resulting meshes have different combinatorics, making the mesh data inherently discontinuous with respect to parameter adjustments. In our work, we address these challenges and develop tools that allow interactive exploration and optimization of parametric CAD data. To achieve interactive rates, we use precomputation on an adaptively sampled grid and propose a novel scheme for interpolating in this domain where each sample is a mesh with different combinatorics. Specifically, we extract partial correspondences from CAD representations for local mesh morphing and propose a novel interpolation method for adaptive grids that is both continuous/smooth and local (i.e., the influence of each sample is constrained to the local regions where mesh morphing can be computed). We show examples of how our method can be used to interactively visualize and optimize objects with a variety of physical properties.
@article{schulz:2017:InstantCAD, author = {Schulz, Adriana and Xu, Jie and Zhu, Bo and Zheng, Changxi and Grinspun, Eitan and Matusik, Wojciech}, title = {Interactive design space exploration and optimization for CAD models}, year = {2017}, issue_date = {August 2017}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {36}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/3072959.3073688}, doi = {10.1145/3072959.3073688}, journal = {ACM Trans. Graph.}, month = jul, articleno = {157}, numpages = {14}, keywords = {simulation, precomputations, parametric shapes, interpolation, CAD}, }
Retrieval on Parametric Shape Collections

Adriana Schulz, Ariel Shamir, Ilya Baran, David I. W. Levin, Pitchaya Sitthi-Amorn, and Wojciech Matusik

ACM Trans. Graph., Jan 2017

Abs DOI Bib

While collections of parametric shapes are growing in size and use, little progress has been made on the fundamental problem of shape-based matching and retrieval for parametric shapes in a collection. The search space for such collections is both discrete (number of shapes) and continuous (parameter values). In this work, we propose representing this space using descriptors that have shown to be effective for single shape retrieval. While single shapes can be represented as points in a descriptor space, parametric shapes are mapped into larger continuous regions. For smooth descriptors, we can assume that these regions are bounded low-dimensional manifolds where the dimensionality is given by the number of shape parameters. We propose representing these manifolds with a set of primitives, namely, points and bounded tangent spaces. Our algorithm describes how to define these primitives and how to use them to construct a manifold approximation that allows accurate and fast retrieval. We perform an analysis based on curvature, boundary evaluation, and the allowed approximation error to select between primitive types. We show how to compute decision variables with no need for empirical parameter adjustments and discuss theoretical guarantees on retrieval accuracy. We validate our approach with experiments that use different types of descriptors on a collection of shapes from multiple categories.
@article{schulz:2017:ParametricRetrieval, author = {Schulz, Adriana and Shamir, Ariel and Baran, Ilya and Levin, David I. W. and Sitthi-Amorn, Pitchaya and Matusik, Wojciech}, title = {Retrieval on Parametric Shape Collections}, year = {2017}, issue_date = {February 2017}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {36}, number = {1}, issn = {0730-0301}, url = {https://doi.org/10.1145/2983618}, doi = {10.1145/2983618}, journal = {ACM Trans. Graph.}, month = jan, articleno = {11}, numpages = {14}, keywords = {parametric designs, Shape retrieval}, }

2016

Computational multicopter design

Tao Du, Adriana Schulz , Bo Zhu, Bernd Bickel, and Wojciech Matusik

ACM Trans. Graph., Dec 2016

Abs DOI Bib

We present an interactive system for computational design, optimization, and fabrication of multicopters. Our computational approach allows non-experts to design, explore, and evaluate a wide range of different multicopters. We provide users with an intuitive interface for assembling a multicopter from a collection of components (e.g., propellers, motors, and carbon fiber rods). Our algorithm interactively optimizes shape and controller parameters of the current design to ensure its proper operation. In addition, we allow incorporating a variety of other metrics (such as payload, battery usage, size, and cost) into the design process and exploring tradeoffs between them. We show the efficacy of our method and system by designing, optimizing, fabricating, and operating multicopters with complex geometries and propeller configurations. We also demonstrate the ability of our optimization algorithm to improve the multicopter performance under different metrics.
@article{du:2016:multicopters, author = {Du, Tao and Schulz, Adriana and Zhu, Bo and Bickel, Bernd and Matusik, Wojciech}, title = {Computational multicopter design}, year = {2016}, issue_date = {November 2016}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {35}, number = {6}, issn = {0730-0301}, url = {https://doi.org/10.1145/2980179.2982427}, doi = {10.1145/2980179.2982427}, journal = {ACM Trans. Graph.}, month = dec, articleno = {227}, numpages = {10}, keywords = {parametric modeling, optimization, fabrication}, }

2014

Design and fabrication by example

Adriana Schulz, Ariel Shamir, David I. W. Levin, Pitchaya Sitthi-amorn, and Wojciech Matusik

ACM Trans. Graph., Jul 2014

Abs DOI Bib

We propose a data-driven method for designing 3D models that can be fabricated. First, our approach converts a collection of expert-created designs to a dataset of parameterized design templates that includes all information necessary for fabrication. The templates are then used in an interactive design system to create new fabri-cable models in a design-by-example manner. A simple interface allows novice users to choose template parts from the database, change their parameters, and combine them to create new models. Using the information in the template database, the system can automatically position, align, and connect parts: the system accomplishes this by adjusting parameters, adding appropriate constraints, and assigning connectors. This process ensures that the created models can be fabricated, saves the user from many tedious but necessary tasks, and makes it possible for non-experts to design and create actual physical objects. To demonstrate our data-driven method, we present several examples of complex functional objects that we designed and manufactured using our system.
@article{schulz:2014:fabByExample, author = {Schulz, Adriana and Shamir, Ariel and Levin, David I. W. and Sitthi-amorn, Pitchaya and Matusik, Wojciech}, title = {Design and fabrication by example}, year = {2014}, issue_date = {July 2014}, publisher = {Association for Computing Machinery}, address = {New York, NY, USA}, volume = {33}, number = {4}, issn = {0730-0301}, url = {https://doi.org/10.1145/2601097.2601127}, doi = {10.1145/2601097.2601127}, journal = {ACM Trans. Graph.}, month = jul, articleno = {62}, numpages = {11}, keywords = {data-driven methods, design, fabrication}, }