This research tackles a low-cost (without expensive datasets) and general strategy to reproduce the plastic deformation of materials with stable physical properties (such as clay) by a robot manipulator. 

In this phase, the target work is making ditches built using human fingers or a stylus with various trajectories, depths, and widths on the surface of silky sand (98% sand + 2% polymer) placed in a box-shaped container (vat), which are observed by the Azure Kinect RGB-D sensor (to acquire 3D point cloud data) and reproduced by a 6-DOF (degree-of-freedom) manipulator (Niryo One robot arm). In other words, the goal is to establish a technology for semi-three-dimensional object deformation (with limited complexity and variation). We have proposed the following three main techniques: 

(1) Labeling 3D surface points: The principal curvatures are calculated, the surface points are classified accordingly, and the labeling is verified by a local data smoothing technique. 

(2) Object shape description: Candidates for spline control points are selected, 2D splines are selected through quality evaluation, and a ditch model described by 2D or 3D splines is obtained using a newly developed ditch width estimation algorithm and a modified RANSAC (random sample consensus) spline fitting algorithm. 

(3) Robot trajectory generation: The splines are defined as modulated sine waves expanded along the spline, and are generated by converting the reference frame. 

These processing steps have been precisely formulated and each part has been compiled into a general-purpose algorithm. Furthermore, appropriate parameter values have been settled for each algorithm by testing various settings. The performance of ditch reproduction by a robot manipulator is tested with various ditches such as geometric patterns and letters, and the similarity of shapes in curvature features and geometric features have been evaluated using the Point Cloud Structural Similarity Measure (PointSSIM). These proposed algorithms and the results of the evaluation experiments have been compiled into a full paper and submitted to an academic journal, which is currently undergoing peer review. 

In the next phase, in order to apply the results so far to a wider variety of object deformations, we will take up plasticine shaping as a more general example of 3D object deformation. Specifically, we will develop the model-free method of "learning from human demonstrations (imitation learning)". To achieve this, we are considering the following algorithms. 

(1) Shape alignment between the initial state and the goal state using 3D point clouds (distinguishing between the parts to be held and the parts to be processed).

(2) Planning robot motion to process the unnecessary parts. 

Based on these concepts, we are currently applying for grant funding.