In object forming tasks, we aim to achieve a robot function that can reproduce the same object forming task simply by observing the shape before and after deformation by a human, without being instructed on the specific steps. This research is concerned with the "learning-based acquisition of robotic skills," but previous research has not fully addressed the following characteristics: (1) the ability to deform complex 3D shapes with a high degree of precision comparable to human work, (2) the ability to work with unknown shapes that have never been handled before, and (3) the ability to reproduce a variety of shapes from a limited number of human demonstrations. Here, we solve these problems with a method that combines learning-based object shape recognition with robot motion planning that learns from human demonstrations. Specifically, we propose a method called the Point Cloud Difference Diffusion Policy (PCDDP).

Here, we deal with object deformation with constant volume (the shape changes when an external force is applied) and plasticity (the shape is maintained even when the external force is removed).

The problem was stipulated as follows: [Target object] Hydrophobic sand (magic sand. It has a smooth texture but can be shaped by gripping with the palm of your hand or pinching with your fingertips, and can be considered a plastically deformable object, such as kinetic sand). [Target task] Shaping the shape of a plastically deformable object such as dough or clay using fingertip movements. As a single-finger task (index finger), three-dimensional objects on a flat surface are shaped primarily by squeezing and stretching. For example, this is the shaping of relatively flat accessories and decorative items such as brooches and pendants. For robotic reproduction, a tabletop six-degree-of-freedom robotic arm (Niryo One, manufactured by Niryo, which we own) is used, and a finger model (index finger model) is attached to the tip of the arm to perform the object deformation task. The task involves deforming a tabletop object with relatively simple movements, but both the object and the deformation are three-dimensional, allowing for basic consideration of logic and algorithms. 

In the future, we plan to expand this to the formation of three-dimensional objects in space using two fingers (index finger and thumb), mainly by pinching, squeezing, and stretching, and we are currently applying for research grants.