Challenges and Outlook in Robotic Manipulation of Deformable Objects
可变形物体机器人操作中的挑战与展望

UNTIL now, object rigidity is one of the common assumptions in robotic grasping and manipulation. Strictly speaking, all objects deform upon force interaction. Rigidity is a valid assumption when object deformation can be neglected in the task. Nevertheless, many objects that need to be manipulated by robots present non-negligible deformation: from micro surgical operation to challenging industrial assemblies.
迄今为止，物体刚性是机器人抓取和操作中的一个常见假设。严格来说，所有物体在受力时都会发生形变。当任务中可以忽略物体形变时，刚性假设是有效的。然而，许多需要由机器人操作的对象表现出不可忽视的形变：从微创手术到具有挑战性的工业装配。

Robots need to be capable of manipulating deformable objects to operate in human environments. This capability
机器人需要能够操作可变形物体，以便在人类环境中工作。这种能力
J. Zhu is with Cognitive Robotics, TU Delft and Honda Research Institute, Europe. j. zhu-3@tudelft.nl
J. Zhu 任职于代尔夫特理工大学认知机器人与本田欧洲研究院。j. zhu-3@tudelft.nl
A. Cherubini is with LIRMM - Université de Montpellier CNRS, 161 Rue Ada, 34090 Montpellier, France. andrea. cherubini@lirmm. fr
A. Cherubini 隶属于 LIRMM - 蒙彼利埃大学 CNRS，地址为法国蒙彼利埃 161 Rue Ada, 34090。邮箱：andrea.cherubini@lirmm.fr
C. Dune is with COSMER laboratory EA 7398, Universié de Toulon, 83130 La Garde, France. claire. dune@univ-tln.fr
C. Dune 隶属于 COSMER 实验室 EA 7398，土伦大学，83130 拉加尔德，法国。claire.dune@univ-tln.fr
D. Navarro-Alarcon is with The Hong Kong Polytechnic University, Department of Mechanical Engineering, Kowloon, Hong Kong. dna@ieee. org
D. Navarro-Alarcon 就职于香港理工大学机械工程系，香港九龙。dna@ieee.org
F. Alambeigi is with the University of Texas at Austin, Austin, USA. farshid.alambeigi@austin.utexas.edu
F. Alambeigi 就职于美国德克萨斯大学奥斯汀分校，奥斯汀，美国。farshid.alambeigi@austin.utexas.edu
D. Berenson is with the University of Michigan, Ann Arbor, MI, USA. berenson@eecs.umich. edu
D. Berenson 就职于美国密歇根州安娜堡的密歇根大学。berenson@eecs.umich.edu
F. Ficuciello is with Università degli Studi di Napoli Federico II, 80125 Napoli, Italy fanny.ficuciello@unina.it
F. Ficuciello 就职于意大利那不勒斯费德里科二世大学，80125 那不勒斯，意大利 fanny.ficuciello@unina.it
K. Harada is with the Osaka University, Japan, and the National Institute of AIST, Japan harada@sys.es.osaka-u.ac.jp
K. Harada 就职于日本大阪大学和日本国家产业技术综合研究所（AIST），电子邮件为 harada@sys.es.osaka-u.ac.jp。
J. Kober is with Cognitive Robotics, TU Delft, the Netherlands J.Kober@tudelft.nl
J. Kober 就职于荷兰代尔夫特理工大学认知机器人实验室，邮箱：J.Kober@tudelft.nl
X. Li is with Department of Automation, Tsinghua University, Beijing, China. xiangli@tsinghua.edu.cn
X. Li 任职于清华大学自动化系，北京，中国。xiangli@tsinghua.edu.cn
J. Pan is with with the Department of Computer Science, The University of Hong Kong, Pok Fu Lam, Hong Kong. jpan@cs.hku. hk
J. Pan 就职于香港大学计算机科学系，香港薄扶林。jpan@cs.hku.hk
W. Yuan is with Robotics Institute, Carnegie Mellon University, Pittsburgh, PA 15213 USA wenzheny@andrew. cmu. edu
W. Yuan 就职于卡内基梅隆大学机器人研究所，位于宾夕法尼亚州匹兹堡，邮编 15213，邮箱 wenzheny@andrew.cmu.edu。
M. Gienger is with Honda Research Institute Europe, Offenbach, Germany Michael.Gienger@honda-ri.de
M. Gienger 就职于本田欧洲研究院，位于德国奥芬巴赫，电子邮件：Michael.Gienger@honda-ri.de

This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible.
本作品已提交 IEEE，可能被出版。版权可能会在无通知的情况下转移，此后该版本可能无法访问。

Fig. 1. Applications involving manipulation of deformable objects. Clockwise from top left: dressing assistance [1], cable harnessing [2], fruit harvesting [3], suturing [4]
图 1. 涉及可变形物体操作的应用。从左上角顺时针方向依次为：穿衣辅助[1]、电缆布线[2]、水果采摘[3]、缝合[4]
would benefit many application fields, while posing fundamental research challenges. In this article, we consider a generalized concept of manipulation where grasping is also part of the task. We will refer to the problem as deformable object manipulation (DOM).
将有益于许多应用领域，同时也提出了基础研究挑战。在本文中，我们考虑了一种广义的操作概念，其中抓取也是任务的一部分。我们将这个问题称为可变形物体操作（DOM）。

The tasks involved in DOM cover a broad spectrum (see Fig. 1). These include: dressing assistance in elderly care, cable harnessing in industrial automation, harvesting and processing fruit and vegetables in agriculture, surgical operations in medical services, to name a few.
DOM 涉及的任务范围广泛（见图 1）。其中包括：老年护理中的穿衣辅助、工业自动化中的线缆整理、农业中的果蔬采摘和加工、医疗服务中的外科手术等。
On the technical side, addressing deformation introduces the following technical challenges:
在技术层面，解决变形问题引入了以下技术挑战：

We believe that overcoming these challenges is not only beneficial to DOM, but can further push towards developing autonomous robots which can operate in unstructured environments. In recent years, there have been a few surveys on robotic manipulation of deformable objects. Some surveys focus on specific areas of DOM. The survey from Jimenez [6] focuses on model-based manipulation planning. More recently, Herguedas et al. [7] review works using multi-robot systems for DOM while the work of [8] considers multimodal sensing. The authors of [9] present the state-of-theart on deformable object modeling for manipulation. There
我们相信，克服这些挑战不仅对 DOM 有益，还能进一步推动开发能在非结构化环境中运行的自主机器人。近年来，已有一些关于可变形物体机器人操作的综述。一些综述专注于 DOM 的特定领域。Jimenez 的综述[6]专注于基于模型的操作规划。最近，Herguedas 等人[7]回顾了使用多机器人系统进行 DOM 的工作，而[8]的工作则考虑了多模态感知。[9]的作者介绍了用于操作的可变形物体建模的最新进展。

Fig. 2. A typical robotic framework for handling deformable objects. In this particular example, the framework addresses wire harness [5].
图 2. 处理可变形物体的典型机器人框架。在这个特定示例中，该框架针对线束[5]。
are also two comprehensive surveys in the area. The survey in [10] reviews and classifies the state-of-the-art according to the object’s physical properties. Lately, [11] reports most recent advances in modeling, learning, perception, and control in DOM.
在该领域也有两项全面的调查。[10]中的调查根据物体的物理特性对最新技术进行了回顾和分类。最近，[11]报告了在 DOM 的建模、学习、感知和控制方面的最新进展。

In contrast with the mentioned surveys, which either focus on reporting the progress of the field or a specific area, this article aims at identifying scientific challenges introduced by object deformations and at projecting crucial future research directions. As DOM is an emerging field of research where there is still much to be done, in this paper, previous works and open problems are given equal weights. In addition, we dedicate one section to discussing practical challenges in various applications of DOM. We believe the paper is a first of its kind, in the field of DOM.
与上述调查相比，这些调查要么侧重于报告该领域的进展，要么侧重于某一特定领域，本文旨在识别由物体变形引入的科学挑战，并预测关键的未来研究方向。由于 DOM 是一个新兴的研究领域，仍有许多工作要做，因此在本文中，先前的工作和未解决的问题被赋予了同等的重要性。此外，我们专门用一节来讨论 DOM 在各种应用中的实际挑战。我们相信，本文是 DOM 领域中的首创。

A robotic framework designed to handle deformable objects usually consists of five key components: gripper and robot design, sensing, modeling, planning and control (See Fig. 2). To position the current research and identify future trends, we conducted a survey on the future perspective of deformable object manipulation ${}^1$${}^1$^(1){ }^{1}. We shared the survey with people working in related field, at various career stages. They were asked to rate the importance and research maturity of each of the five identified key components, from 1 to 4 , with 1 being not important/low maturity and 4 being very important/high maturity. We received 31 answers that are summarized on Fig. 3.
一个旨在处理可变形物体的机器人框架通常由五个关键组件组成：夹持器和机器人设计、感知、建模、规划和控制（见图 2）。为了定位当前研究并识别未来趋势，我们对可变形物体操作的未来前景进行了调查 ${}^1$${}^1$^(1){ }^{1} 。我们将调查分享给相关领域的不同职业阶段的工作人员。他们被要求对五个已确定的关键组件的重要性和研究成熟度进行评分，从 1 到 4，其中 1 表示不重要/低成熟度，4 表示非常重要/高成熟度。我们收到了 31 份回答，总结在图 3 中。

We consider the promising direction of research as the ones that have the highest significance and the lowest research maturity. Based on the survey, sensing is the most promising one among all subareas. This is probably due to current booming trend in Deep Learning has offered many new methods for processing the sensory data. In addition, sensing is the prerequisite for subsequent steps such as modeling, planning and control.
我们认为最具研究前景的方向是那些具有最高重要性和最低研究成熟度的领域。根据调查，感知是所有子领域中最有前景的一个。这可能是由于当前深度学习的蓬勃发展，为处理感知数据提供了许多新方法。此外，感知是后续步骤（如建模、规划和控制）的先决条件。

Accordingly, the following sections of this paper each present one of these five research directions. In each section, we review recent works in the field and then comment the outlook and challenges ahead. Then, Sect. VII tries to provide
因此，本文的以下部分分别介绍了这五个研究方向。在每一部分中，我们回顾了该领域的最新工作，并对未来的前景和挑战进行了评论。然后，第七节试图提供

(a) Highest qualifications of the respondents.
(a) 受访者的最高学历。

(b) Means and variances of Importance and research maturity ratings of each key component.
(b) 每个关键组件的重要性和研究成熟度评分的均值和方差。
Fig. 3. Summary of the outcomes of the survey on DOM. We received in total 31 answers. The respondents cover different level of qualifications ranging from master students to full professors.
图 3. DOM 调查结果总结。我们总共收到了 31 份回复。受访者涵盖了从硕士研究生到正教授等不同资历水平。
a link from research to practical applications in the context of DOM. Finally, we summarize key messages in Sect. VIII.
在 DOM 的背景下，从研究到实际应用的链接。最后，我们在第八节总结了关键信息。

Does manipulation of deformable objects demand specific grippers as compared to manipulation of rigid objects? Generally, yes (see Fig. 4). Unlike rigid objects (which are mostly handled by standard grippers), deformable objects are handled with custom (and often designed ad-hoc) grippers, e.g. a 3D printed gripper that enables cable sliding [5], a flat clip for holding towels [12], a cylindrical tool for pushing and tapping plastic materials [13], a soft hand for manipulating organs [14]. Such diversity in grippers is a result of the large variety
与刚性物体的操作相比，可变形物体的操作是否需要特定的夹持器？一般来说，是的（见图 4）。与刚性物体（通常由标准夹持器处理）不同，可变形物体通常使用定制（通常是专门设计的）夹持器进行处理，例如，用于电缆滑动的 3D 打印夹持器[5]、用于固定毛巾的扁平夹子[12]、用于推动和敲击塑料材料的圆柱形工具[13]、用于操作器官的软手[14]。这种夹持器的多样性是由于可变形物体的种类繁多所致。
of deformable objects, which require different actions during manipulation. To avoid designing task-specific grippers for DOM, human-like dexterity and compliance is desired. Recent works in this directions consider compliant design [15], [16] and show good potential for DOM tasks.
可变形物体的操作需要不同的动作。为了避免为 DOM 设计特定任务的夹持器，需要具备类似人类的灵活性和顺应性。最近在这方面的研究考虑了顺应性设计[15], [16]，并显示出在 DOM 任务中的良好潜力。

As for the robot itself, it is rigid in most works. In some cases, as in the surgical application showcased in [17] (Fig. 4, bottom right), both robot and object are deformable to ensure safety of manipulation.
至于机器人本身，在大多数作品中它是刚性的。在某些情况下，如[17]中展示的手术应用（图 4，右下角），机器人和物体都是可变形的，以确保操作的安全性。

Fig. 4. Various robot grippers for DOM. Clockwise from top left: a tool for pushing and tapping on plastic materials [13], flat clips for holding a towel [12], a gripper allowing a cable to slide [5], a soft continuum manipulator interacting with a deformable material [17] and a soft hand for manipulating organs [14].
图 4. 用于 DOM 的各种机器人夹持器。从左上角顺时针方向依次为：用于推动和敲击塑料材料的工具[13]，用于固定毛巾的扁平夹子[12]，允许电缆滑动的夹持器[5]，与可变形材料交互的软连续机械臂[17]，以及用于操作器官的软手[14]。

Improving dexterity is core to robot manipulation. The improvement can come from different research domains, such as accurate in-hand sensing or robust control, two aspects which we will detail in Sections III and VI, respectively. In this section, our focus is on gripper/robot hardware aspects.
提高灵巧性是机器人操作的核心。这种改进可以来自不同的研究领域，例如精确的手内感知或鲁棒控制，我们将在第三和第六节中分别详细讨论这两个方面。在本节中，我们的重点是夹持器/机器人硬件方面。

One way of achieving such dexterity is to reproduce by design the most dexterous gripper - the human hand. An open question is whether anthropomorphic design is in itself the optimal solution in all cases, especially in the context of DOM.
实现这种灵活性的一种方法是，通过设计复制最灵活的抓取器——人手。一个悬而未决的问题是，拟人化设计本身是否在所有情况下都是最优解决方案，特别是在 DOM 的背景下。

While having one dexterous gripper which can handle a variety of DOM tasks is appealing, it should be noted that additional constraints need to be considered in the design process, for hygiene/safety in tasks such as food handling or surgery. For instance, for surgical applications we are limited by biocompatibility of the materials and actuators and by the reduced available space in minimally invasive surgery. In theses cases, designing task-specific grippers is more appropriate. Non-anthropomorphic soft grippers are another emerging area of research [18]. These grippers are promising, to overcome the challenges associated with traditional fingered grippers in grasping rigid objects; yet, to date, their application to DOM receives little attention.
虽然拥有一个能够处理各种 DOM 任务的灵巧夹爪很吸引人，但需要注意的是，在设计过程中还需要考虑额外的限制，例如在食品处理或手术等任务中的卫生/安全要求。例如，对于手术应用，我们受到材料和执行器的生物相容性以及微创手术中可用空间减少的限制。在这些情况下，设计特定任务的夹爪更为合适。非拟人化的软夹爪是另一个新兴的研究领域[18]。这些夹爪在克服传统指状夹爪在抓取刚性物体方面的挑战方面很有前景；然而，迄今为止，它们在 DOM 中的应用很少受到关注。

Otherwise, one may use a standard gripper, and provide the robot with suitable tools to be grasped and used according to the type of task at stake. This demands breakthroughs on the algorithmic side, to make the robot capable of reasoning on the proper tools for different tasks. Training the robot to have task-specific reasoning will enhance autonomy and make robots realize more complex tasks.
否则，可以使用标准的夹持器，并为机器人提供适合的工具，以便根据具体任务类型进行抓取和使用。这要求在算法方面取得突破，使机器人能够对不同任务所需的合适工具进行推理。训练机器人具备任务特定的推理能力将增强其自主性，并使机器人能够完成更复杂的任务。

Another area worth investigating is that of soft robots/grippers, since these have great potential for manipulating fragile materials, such as organs or food, or for collecting biological samples or fruits (see Fig. 5). While traditional rigid robots need to exhibit compliant behavior when interacting with these objects, the inherent compliance of soft robots makes the task safe. This unconventional paradigm of using soft robots to manipulate soft objects will bring new challenges in modeling and control as both the robot and the object are under-actuated and difficult to model. One pioneering work in this direction is [21], which adapts the finite element modeling (FEM) based inverse soft robot model with contact handling (proposed in [22]) for deformable objects manipulation using soft robots.
另一个值得研究的领域是软体机器人/抓手，因为它们在操纵脆弱材料（如器官或食物）或收集生物样本或水果方面具有巨大潜力（见图 5）。传统的刚性机器人在与这些物体交互时需要表现出顺应性行为，而软体机器人固有的顺应性使得任务更加安全。这种使用软体机器人操纵软物体的非常规范式将在建模和控制方面带来新的挑战，因为机器人和物体都是欠驱动的，难以建模。在这方面的一项开创性工作是[21]，它采用了基于有限元建模（FEM）的逆向软体机器人模型，并结合了接触处理（在[22]中提出），用于使用软体机器人操纵可变形物体。

An interesting research question to consider is whether methods can be transferred from one field to the other. To be more specific, can methods for controlling/modeling soft robots be applied to manipulating deformable objects and vice versa? If so, as a community, it may be valuable to obtain a unified approach for working with both soft robots and deformable objects.
一个有趣的研究问题是，方法是否可以从一个领域转移到另一个领域。更具体地说，控制/建模软机器人的方法是否可以应用于操纵可变形物体，反之亦然？如果可以，作为一个社区，获得一种统一的方法来处理软机器人和可变形物体可能是有价值的。

In this section, we consider visual, tactile and force sensing for DOM. Existing research relies on these three modes to estimate the state of deformable objects. In most cases, vision provides global information about shapes on a large scale, while force and tactile provide local information on both shape and contact. At the end of this section, we also discuss the research in contrast with this common practice, where global deformation properties are recovered using tactile sensing. It should also be noted that force information is particularly important in industrial settings, e.g., for assembly [23], [24]. Vision is used in tasks such as rope manipulation [25], [26] or cloth unfolding [27], [28], where the object exhibits large global deformation. In these works, configurations of deformable objects were obtained from raw image readings. Although vision offers a global perspective of the object configuration, visual data can be noisy in unstructured environments, it is then important to manage occlusions [12], [29]. Most of above-mentioned works are based on 2D vision, 3D perception of deformable objects is more challenging. Existing works employ FEM [30] or a combination of Growing Neural
在本节中，我们考虑将视觉、触觉和力觉传感用于可变形物体的操作。现有研究依赖这三种模式来估计可变形物体的状态。在大多数情况下，视觉提供大规模形状的全局信息，而力和触觉则提供形状和接触的局部信息。在本节末尾，我们还讨论了与这种常见做法相对的研究，其中使用触觉传感来恢复全局变形特性。还应注意的是，在工业环境中，如装配[23]、[24]，力信息尤为重要。视觉被用于诸如绳索操作[25]、[26]或布料展开[27]、[28]等任务中，这些任务中物体表现出较大的全局变形。在这些工作中，可变形物体的配置是从原始图像读数中获得的。尽管视觉提供了物体配置的全局视角，但在非结构化环境中，视觉数据可能含有噪声，因此管理遮挡[12]、[29]非常重要。上述大多数工作基于 2D 视觉，而对可变形物体的 3D 感知更具挑战性。 现有工作采用 FEM [30]或结合 Growing Neural

Fig. 5. Two examples of interaction with fragile objects, which could benefit from the use of soft robots.
图 5. 与易碎物品交互的两个例子，这些场景可以从软机器人的使用中受益。

Gas and Particle Graph Networks [31] for better tracking the deformation. In a more recent study [32], it has been shown that a deep convolutional neural network for processing vision data can be used with small variations to process tactile data for deformable objects recognition.
气体和粒子图网络 [31] 用于更好地跟踪变形。在最近的一项研究 [32] 中，已经证明，用于处理视觉数据的深度卷积神经网络只需稍作修改即可用于处理触觉数据，以识别可变形物体。

Objects made of soft materials, such as human tissues and fruits, have a special force-displacement correlation upon contact. As a result, tactile sensing can be used to estimate the stiffness. In [33], the GelSight [34], a vision-based highresolution tactile sensor, measures the 3D geometry of the contact surface, and the normal/shear forces.
由软材料制成的物体，如人体组织和水果，在接触时具有特殊的力-位移相关性。因此，触觉感知可用于估计刚度。在[33]中，GelSight [34]这种基于视觉的高分辨率触觉传感器，测量了接触表面的 3D 几何形状以及法向/剪切力。
Note that the division of vision for global deformation and tactile sensing for local deformation is not absolute. The authors of [35] present to use vision to estimate the local deformation of objects during grasping, and classify objects accordingly. In [36], high-resolution tactile sensing is used to estimate the physical properties of clothing materials through squeezing, assuming the robot can learn from the data about indicating global properties of clothing according to a local sampling point. In [37] an example of servoing along a cable based on high-resolution tactile sensing is presented. Although vision is not used, the precise measurement of the local cable shape provides enough information to guide the robot motion on a small scale.
请注意，全局变形的视觉划分和局部变形的触觉感知并不是绝对的。[35]的作者提出使用视觉来估计抓取过程中物体的局部变形，并据此对物体进行分类。在[36]中，高分辨率触觉感知被用来通过挤压估计衣物材料的物理特性，假设机器人可以从数据中学习根据局部采样点指示衣物的全局特性。[37]中展示了一个基于高分辨率触觉感知沿电缆伺服控制的例子。虽然未使用视觉，但电缆局部形状的精确测量提供了足够的信息来指导机器人进行小规模运动。

Here, the main challenges are: selecting appropriate sensors for the DOM task and using the measurements to obtain meaningful object representations.
在这里，主要挑战是：为 DOM 任务选择合适的传感器，并利用测量结果获得有意义的目标表示。
Considering the high number of degrees of freedom (DoF) of the deformable bodies, fusion of different sensing modalities (vision, force and tactile) may be a promising direction to pursue in future research.
考虑到可变形体的高自由度（DoF），融合不同的传感模式（视觉、力和触觉）可能是未来研究中值得探索的一个有前景的方向。

Another research question to be answered is: what yields a good representation of the object configuration? We (acknowledgedly) do not have a complete answer to this; rather, we will elaborate on considerations when designing the representation.
另一个需要回答的研究问题是：什么能产生良好的对象配置表示？我们（承认）对此没有完整的答案；相反，我们将详细阐述设计表示时的考虑因素。

The representation need to be robust to noise and useful for reconstructing the objects’ configuration - even when data are partially unavailable. In vision, the most common noise is occlusion. How to generate a meaningful representation of these objects under self occlusion is still an open problem in research. For rigid objects, one can carefully design the environment to avoid it. For deformable objects that exhibit large global deformation such as clothes, bed sheets etc, self occlusion is inevitable during manipulation. A promising direction to deal with occlusion and noise is using active/interactive perception. With vision data from different perspectives, we might be able to reconstruct the object’s configuration accurately even under occlusion and noise.
表示需要对噪声具有鲁棒性，并且对于重建物体配置有用——即使在数据部分不可用的情况下。在视觉中，最常见的噪声是遮挡。如何在自遮挡下生成这些物体的有意义的表示仍然是研究中的一个开放问题。对于刚性物体，可以精心设计环境以避免这种情况。对于表现出大范围全局变形的可变形物体，如衣物、床单等，在操作过程中自遮挡是不可避免的。处理遮挡和噪声的一个有前景的方向是使用主动/交互感知。通过来自不同视角的视觉数据，即使在遮挡和噪声的情况下，我们也可能能够准确地重建物体的配置。

Apart from the above mentioned challenges, choosing a good representation also involves leveraging two aspects:
除了上述挑战之外，选择一个好的表示还涉及利用两个方面：

Usually the trade-off depends on the task, relies on human intuition and involves a trial and error process. In end-to-end reinforcement learning settings, sensory data can be mapped
通常，权衡取决于任务，依赖于人类直觉，并涉及试错过程。在端到端的强化学习设置中，感官数据可以被映射
directly to robot actions without explicit feature representations [38]. Human demonstrations can be used for making end-to-end learning more efficient. One example is reported in [39]. The authors use an improved version of Deep Deterministic Policy Gradients, trained with 20 demonstrations, to make robots manipulate cloth. However, since such settings often require a manually designed cost/reward function for learning, human demonstrations in this context can also be used for recovering the reward, with inverse reinforcement learning.
直接转化为机器人动作，而无需显式的特征表示[38]。人类示范可用于使端到端学习更加高效。[39]中报告了一个例子。作者使用改进版的深度确定性策略梯度，通过 20 次示范进行训练，使机器人能够操作布料。然而，由于此类设置通常需要手动设计用于学习的成本/奖励函数，因此在这种背景下的人类示范也可用于通过逆强化学习恢复奖励。

For robots to perform deformation tasks using sensory data, we need a model that captures the relationship between sensor information and robot motion. A linear model characterized by Young’s modulus can be employed for describing elastic deformation. The two other classes of deformation are: plastic, and elasto-plastic deformations. This classification serves well. Yet, since the model should be used for control, in this section, we prefer to distinguish between local and global models a taxonomy which has clearer implications for control. We introduce the corresponding research and - at the end of the section - we discuss the limitations of these models and present works that address them.
为了让机器人利用传感数据执行形变任务，我们需要一个模型来捕捉传感器信息与机器人运动之间的关系。描述弹性形变时，可以采用以杨氏模量为特征的线性模型。另外两类形变是：塑性形变和弹塑性形变。这种分类方式十分适用。然而，由于该模型需用于控制，在本节中，我们更倾向于区分局部模型与全局模型，这一分类法对控制有着更明确的指导意义。我们介绍了相关研究，并在本节末尾讨论了这些模型的局限性，以及针对这些局限性的研究工作。

Most local models approximate the perception/action relationship via a Jacobian Matrix (called Interaction Matrix in visual servoing). Such a model is linear and can be computed in real-time with a small amount of data. Yet, since it is a local model, it should be continuously updated during task execution. Model updating methods include: Broyden rule [17], receding horizon adaption [40], local gradient descent [41], QP-based optimization [42], and Multi-armed Banditbased methods [43]. Another advantage of the Jacobian model is that one can design a simple controller by inverting it. However, since this controller is local, it should operate via a series of intermediate target shapes [40], [42].
大多数局部模型通过雅可比矩阵（在视觉伺服中称为交互矩阵）来近似感知/动作关系。这种模型是线性的，可以用少量数据实时计算。然而，由于它是局部模型，因此在任务执行期间应不断更新。模型更新方法包括：Broyden 规则[17]、滚动时域自适应[40]、局部梯度下降[41]、基于 QP 的优化[42]以及多臂赌博机方法[43]。雅可比模型的另一个优势是可以通过其逆矩阵设计一个简单的控制器。然而，由于该控制器是局部的，因此应通过一系列中间目标形状进行操作[40]，[42]。
On the other hand, global models can be approximated with Finite Element Methods [44] and also (deep) neural networks. In contrast to simple linear models, (D)NN-based approaches benefit from stronger representation power, in terms of accuracy and robustness [45]. Moreover, they can incorporate physics models and reason about object interaction [46]. These models can approximate highly nonlinear systems and have a larger validity range, solving (to some extent) the locality issue of the linear models. Nevertheless, these complex nonlinear representations demand large amounts of data (which might not be available in some cases).
另一方面，全局模型可以通过有限元方法[44]以及（深度）神经网络来近似。与简单的线性模型相比，基于（深度）神经网络的方法在准确性和鲁棒性方面受益于更强的表示能力[45]。此外，它们可以整合物理模型并推理物体间的相互作用[46]。这些模型能够近似高度非线性系统，并具有更大的有效范围，从而在一定程度上解决了线性模型的局部性问题。然而，这些复杂的非线性表示需要大量数据（在某些情况下可能无法获得）。

Yet, whether we use analytical or learned models, their predictive power will be limited. They are either specialized to some class of tasks or learned from a set of training data. Especially for the learned models, we can never hope to collect enough data to produce an accurate model in the entire state space (which is high dimensional). Thus [47] and [48] have developed methods to reason about the validity of a (learned) model for a given state and action, and have used these methods to reason about model uncertainty in planning and control. However, when the model is not precise, a replanning/recovery might be desirable. The authors of [49]
然而，无论我们使用分析模型还是学习模型，它们的预测能力都是有限的。它们要么专门针对某些类别的任务，要么是从一组训练数据中学习而来的。特别是对于学习模型，我们永远无法希望收集到足够的数据来在整个状态空间（高维）中生成一个准确的模型。因此，[47]和[48]开发了方法来推理（学习）模型在给定状态和动作下的有效性，并利用这些方法来推理规划和控制中的模型不确定性。然而，当模型不精确时，重新规划/恢复可能是可取的。[49]的作者们
introduces two neural networks for learning and re-planning the motion when the model is unreliable.
引入了两个神经网络，用于在模型不可靠时学习和重新规划运动。

The complexity of modeling is manifested in the lack of simulators. While most existing robotic simulators are capable of producing rigid body kinematics and dynamic behaviours, only a fraction of them can handle deformation. One recent work, Softgym [50] was proposed for bench-marking DOM based on Nvidia Flex. In the soft robotics community, SOFA [51] and Chainqueen [52] are example simulators. In Sect. II-B, we considered the interaction between soft robots and deformable objects. Thus, a unified simulator that is able to handle soft robots and objects, and model their interaction might be desirable.
建模的复杂性体现在缺乏模拟器上。虽然大多数现有的机器人模拟器能够生成刚体运动学和动态行为，但只有少数能够处理变形。最近的一项工作，Softgym [50]，被提出用于基于 Nvidia Flex 的 DOM 基准测试。在软体机器人社区中，SOFA [51]和 Chainqueen [52]是示例模拟器。在第二节 B 部分，我们考虑了软体机器人与可变形物体之间的交互。因此，一个能够处理软体机器人和物体并模拟它们交互的统一模拟器可能是理想的。

When choosing a model for control, one challenge of datadriven deformation modeling is to balance region of validity with number of data required for training. One possible direction is to combine a simple model with a complex nonlinear model to form a hierarchical model. An example of such structures is exploited in [53] for robust in-hand manipulation. For DOM tasks, we can have a linear model at lower level, and a (D)NNs learning the full model at higher level. The lower level model can be learned in few iterations to enable instant interaction between robot and object. The higher level (D)NN can collect data and improve the model to enhance global convergence.
在选择控制模型时，数据驱动变形建模的一个挑战是平衡有效区域与训练所需数据量。一个可能的方向是将简单模型与复杂的非线性模型结合，形成分层模型。例如，[53]中利用这种结构实现了鲁棒的掌内操作。对于 DOM 任务，我们可以在较低层次使用线性模型，在较高层次使用(D)NN 学习完整模型。较低层次的模型可以在几次迭代中学习，以实现机器人与物体之间的即时交互。较高层次的(D)NN 可以收集数据并改进模型，以增强全局收敛性。

Planning aims at finding a sequence of valid (robot/object) configurations and contributes to solving the problem of limited validity of local models, as discussed in Sect. IV.
规划旨在寻找一系列有效的（机器人/物体）配置，并有助于解决第 IV 节中讨论的局部模型有效性有限的问题。

Planners can operate in the objects’ configuration space, and sometimes rely heavily on physic-based simulation. While the obtained plan can be visually plausible, it may be unrealizable for a specific object. Recently, McConachie et. al. presented a framework which combines global planning without physics simulation, with local control [54]. For an elastic object, considering its energy is another way to do planning; in this direction, Ramirez-Alpizar et al. [55] proposed a dual-arm manipulation planner optimizing the elastic energy, for elastic ring-shaped objects manipulation. For DOM tasks involving multiple robots, planning is important for coordination. Alonso-Mora et al. employed a distributed receding horizon planner for transporting tasks that require multiple robots [56]. More recently, [57] learns a latent representation for semantic soft object manipulation that enables (quasi) shape planning with deformable objects.
规划器可以在物体的配置空间中操作，有时严重依赖基于物理的模拟。虽然获得的计划在视觉上可能是合理的，但对于特定物体来说可能是无法实现的。最近，McConachie 等人提出了一个框架，该框架结合了无需物理模拟的全局规划和局部控制[54]。对于弹性物体，考虑其能量是另一种进行规划的方式；在这个方向上，Ramirez-Alpizar 等人[55]提出了一个优化弹性能量的双臂操作规划器，用于弹性环形物体的操作。对于涉及多个机器人的 DOM 任务，规划对于协调至关重要。Alonso-Mora 等人采用了一种分布式滚动时域规划器，用于需要多个机器人进行运输的任务[56]。最近，[57]学习了一种用于语义软物体操作的潜在表示，使得可变形物体的（准）形状规划成为可能。

With Learning from Demonstration (LfD), the robot can be trained to manipulate deformable objects by an expert (usually a human). LfD encodes the robot trajectory and interaction force from human demonstrations [58], thus avoiding explicitly planning the motion. More recently, Wu et al. have proposed a reinforcement learning scheme for DOM, which does not require initial demonstrations [59].
通过从示范中学习（LfD），机器人可以由专家（通常是人类）训练来操作可变形物体。LfD 从人类示范中编码机器人轨迹和交互力[58]，从而避免了显式规划运动。最近，Wu 等人提出了一种用于 DOM 的强化学习方案，该方案不需要初始示范[59]。

A rigid object configuration can be described in space with 6 DoF, whereas a deformable object configuration has a much higher number of DoF. To address this from the sensing algorithm side, one can find a compact representation from sensory data, as discussed in Sect. III-B. An alternative, which receives much less attention, is the use of environmental contacts to constrain some DoF of deformable objects. Examples include the use of contact points in cable harness or that of flat surfaces when folding clothes. We argue that instead of planning to avoid contacts as most planners do, for deformable objects, we need to plan to make contact, since this constrains the configuration, and therefore simplifies the task.
一个刚性物体的构型可以在空间中用 6 个自由度（DoF）来描述，而可变形物体的构型则具有更多的自由度。为了从感知算法的角度解决这个问题，可以从传感器数据中找到一种紧凑的表示，如第 III-B 节所讨论的。另一种较少受到关注的方法是使用环境接触来约束可变形物体的某些自由度。例如，在电缆束中使用接触点或在折叠衣物时使用平坦表面。我们认为，对于可变形物体，与其像大多数规划器那样计划避免接触，不如计划进行接触，因为这可以约束构型，从而简化任务。

Planning to grasp the correct point is often crucial in DOM tasks. For instance, grasping at convex vertices of the clothes guarantees stability and facilitates the task [60]. Re-grasp planning is highly relevant when considering tasks which require multiple robotic arms. Additional challenges come from perception, since as soon as the robot releases one or more grasp(s), the object is likely to change its configuration. We rely on sensing to track configuration changes and then plan accordingly.
在 DOM 任务中，计划抓住正确点通常至关重要。例如，抓住衣物的凸顶点可以保证稳定性并促进任务完成[60]。在考虑需要多个机械臂的任务时，重新抓取规划高度相关。感知方面带来了额外的挑战，因为一旦机器人释放一个或多个抓取点，物体的配置很可能会发生变化。我们依靠传感来跟踪配置变化，然后相应地进行规划。

Another important future work in planning is reasoning about a deformable object at a semantic level. What does it mean for a cloth to be folded? What does it mean for an object to be wrapped in paper? We cannot manually specify all the configurations of the deformable object to use as goals in these kinds of tasks. Instead, we need a way to learn the meaning of semantic concepts, such as folded or wrapped, so that we can determine if a given configuration of the object is a valid goal.
规划领域的另一项重要未来工作是在语义层面上对可变形物体进行推理。布被折叠意味着什么？物体被纸包裹意味着什么？我们无法手动指定可变形物体的所有配置作为这些任务的目标。相反，我们需要一种方法来学习语义概念（如折叠或包裹）的含义，以便确定给定物体配置是否是一个有效的目标。

Control aims at designing inputs for the robot to realize the planned motion. The type of controllers is decided usually by the task. For instance, the authors employed a datadriven model predictive control [61] for cutting considering its predictive nature and the lower demand for manual tuning. For safe interaction in minimally invasive surgery, the authors of [62] used a fuzzy compensator with impedance control. For controlling large deformation, Aranda et al., proposed a Shape-from-Template algorithm concerning its low dimensional representation (using the template) and robustness against occlusion [63].
控制旨在为机器人设计输入以实现计划中的运动。控制器的类型通常由任务决定。例如，作者采用了数据驱动的模型预测控制[61]进行切割，考虑到其预测性质和对手动调谐的较低要求。为了在微创手术中实现安全交互，[62]的作者使用了带有阻抗控制的模糊补偿器。为了控制大变形，Aranda 等人提出了 Shape-from-Template 算法，考虑到其低维表示（使用模板）和对遮挡的鲁棒性[63]。

A number of works focus on shape control. While global models directly map sensor data to robot motion, local models must be inverted to design the robot motion controller (see Sec. IV). Several applications of the control scheme for robotic manipulation of deformable objects can be found in 3C manufacturing [64], [65], where vision-based controllers were proposed to drive the robot to automatically grasp/contact the deformable object, then carry out the task of active deformation or separation/sorting. Other works consider the concept of diminishing rigidity to do deformation control [66], [67].
许多研究聚焦于形状控制。全局模型直接将传感器数据映射到机器人运动，而局部模型则必须经过反演以设计机器人运动控制器（见第 IV 节）。在 3C 制造中，可以找到机器人操控可变形物体控制方案的多种应用[64]，[65]，其中提出了基于视觉的控制器，以驱动机器人自动抓取/接触可变形物体，进而执行主动变形或分离/分类任务。其他研究则考虑采用减小刚度的概念来进行变形控制[66]，[67]。

Feedback control has been commonly used in most DOM works, by referring to the state of the object, to achieve the task. Note that such state is retrieved from the output of its deformation model and measured with sensors, and that output and state do not necessarily have the same representation and dimension. Furthermore, we can distinguish between modelbased and model-free control. Due to the complexity of modeling the deformation, when using the model to derive control policies, the controller has to take into account that the model will be inaccurate or even wrong.
反馈控制在大多数 DOM 工作中被普遍使用，通过参考对象的状态来实现任务。需要注意的是，这种状态是从其变形模型的输出中检索的，并通过传感器进行测量，且输出和状态不一定具有相同的表示和维度。此外，我们可以区分基于模型的控制和无模型控制。由于建模变形的复杂性，当使用模型来推导控制策略时，控制器必须考虑到模型可能不准确甚至错误。
Model-free approaches do not require information about the deformation parameters or the structure of the deformation model. Examples include LfD or (Deep) reinforcement learning, where the challenges are: efficient use of data, and policy generalization. To address these issues, we can combine the offline and online learning methods. In the offline phase, the supervised network can be trained to estimate the model, by collecting pairs of a series of predefined inputs (e.g., the velocity of the robot end-effector) and the deformation of the object. The estimated model in the offline phase can be further updated online during the control task with adaption techniques (e.g., the adaptive NNs), to compensate the errors due to insufficient training in the offline phase or the changes of the deformation model. Hence, both complement each other.
无模型方法不需要关于变形参数或变形模型结构的信息。例如，LfD 或（深度）强化学习，其中的挑战是：数据的高效使用和策略的泛化。为了解决这些问题，我们可以结合离线和在线学习方法。在离线阶段，可以通过收集一系列预定义输入（例如，机器人末端执行器的速度）和物体变形的对来训练监督网络以估计模型。离线阶段估计的模型可以在控制任务期间通过适应技术（例如，自适应神经网络）在线更新，以补偿由于离线阶段训练不足或变形模型变化引起的误差。因此，两者相辅相成。

When multiple features on the deformable object are controlled in parallel, the system becomes under-actuated, with less control inputs than error outputs. Then, the robot controller should be able to deal with the conflicts between multiple features or decouple the control of multiple features in a sequential manner, to guarantee controllability.
当可变形物体上的多个特征被并行控制时，系统变得欠驱动，控制输入少于误差输出。因此，机器人控制器应能够处理多个特征之间的冲突，或以顺序方式解耦多个特征的控制，以确保可控性。

In addition, due to the deformation during control, the contact between robot end-effector and deformable object may not always be maintained. Most existing systems require a certain level of human assistance to initiate the contact or to re-establish it, if it is lost during the task. To improve autonomy, the robot controller should automatically grasp or touch the object first, whenever physical contact is lost, laying the foundation of the subsequent manipulation task. Such a capability would allow the robot to effectively deal with the unforeseen changes due to deformation.
此外，由于控制过程中的变形，机器人末端执行器与可变形物体之间的接触可能无法始终保持。大多数现有系统需要一定程度的人工协助来启动接触或在任务过程中重新建立接触。为了提高自主性，机器人控制器应在物理接触丢失时自动抓取或触摸物体，为后续的操纵任务奠定基础。这种能力将使机器人能够有效应对因变形引起的意外变化。

In previous sections, we centered our discussions from a scientific point of view, here, we instead discuss challenges in various applications where DOM can be translated to solutions.
在之前的章节中，我们从科学的角度集中讨论了相关议题，而在这里，我们将转而探讨 DOM 在各类应用中转化为解决方案时所面临的挑战。

Automatic laundry: A typical domestic application of DOM is laundry folding. A Tokyo-based company unveiled its prototype laundry-folding robot in 2015 (Fig. 6a). However, the company was announced bankrupt in 2019 due to lack of funding for development and difficulties in improving the robot to reach a satisfactory level [68]. Although cloth folding has been tackled in a few previous research [69]-[72], it remains largely a laboratory product (limited to structured environments, certain types of the clothes, etc). Commercializing the technology seems requiring a substantial efforts.
自动洗衣：DOM 的一个典型家用应用是衣物折叠。一家位于东京的公司于 2015 年展示了其原型衣物折叠机器人（图 6a）。然而，由于缺乏开发资金和难以将机器人改进到令人满意的水平，该公司于 2019 年宣布破产[68]。尽管之前有几项研究已经探讨了衣物折叠问题[69]-[72]，但它主要仍是一个实验室产品（仅限于结构化环境、特定类型的衣物等）。商业化这项技术似乎需要大量的努力。

Assistive dressing: Robotic dressing assistance has the potential to become an important technology due to the pressing needs for ageing society support. Research can roughly be categorized into simulation-based learning [73], [74] and imitation learning [75] approaches. Examples are dressing support for shoes [76], shirts [77]-[79] and pants. However, several technical and societal challenges have to be addressed before robot-assisted dressing will become a broadly used DOM technology: physical safety for the human, modeling and prediction of the human-robot interaction, robustness for large variations of geometric and dynamic properties of textiles, low-cost high-reliable robot hardware, human acceptance of such technologies.
辅助穿衣：由于老龄化社会支持的迫切需求，机器人穿衣辅助技术有潜力成为一项重要技术。研究大致可分为基于模拟的学习[73]、[74]和模仿学习[75]方法。例如，鞋子[76]、衬衫[77]-[79]和裤子的穿衣辅助。然而，在机器人辅助穿衣成为广泛应用的 DOM 技术之前，必须解决一些技术和社会挑战：人类的物理安全、人机交互的建模和预测、对纺织品几何和动态特性大变化的鲁棒性、低成本高可靠性的机器人硬件、人类对此类技术的接受度。

Surgical robotics: Soft tissue manipulation is mainly performed with tele-operation solely using visual feedback. Autonomous manipulation, however, still has a long way to go and demands developing various DOM hardware and software (Fig. 6d). The biggest concern for an autonomous solution is the safety of operation. A soft robot with intrinsic compliance will probably enhance the safety.
手术机器人：软组织的操作主要依赖于仅使用视觉反馈的远程操作。然而，自主操作还有很长的路要走，需要开发各种 DOM 硬件和软件（图 6d）。自主解决方案的最大关注点是操作的安全性。具有内在顺应性的软机器人可能会提高安全性。

Food production & Retail: Handling deformable objects is a major challenge in the whole chain from production to sales. In an agricultural setting, automated harvesting of fruits and vegetables requires interactions with deformable objects that are at the same time easy to damage, which immediately decreases their value and shelf live. Frequently, these products also undergo an intermediate processing step (e.g., filleting and packaging meat). More generally, deformable products (e.g., everything packaged in flexible bags, (Fig. 6c)) need to be handled in warehouses, in order picking, and in restocking. Solutions for specific applications and products have been developed, but more complex objects and operations still are frequently handled by human workers.
食品生产与零售：在从生产到销售的整个链条中，处理可变形的物体是一个主要挑战。在农业环境中，自动采摘水果和蔬菜需要与容易损坏的可变形物体进行交互，这会立即降低其价值和保质期。通常，这些产品还会经过中间加工步骤（例如，切片和包装肉类）。更广泛地说，可变形的产品（例如，所有用柔性袋包装的物品，（图 6c））需要在仓库、订单拣选和补货中进行处理。针对特定应用和产品的解决方案已经开发出来，但更复杂的物体和操作仍然经常由人工处理。

Marine robotics: Underwater grasping has been led by oil and gas industry for decades, resulting in heavy machines with strong grippers for inspection and maintenance tasks (Fig. 6e). Gradually the demands turned to more detailed tasks in marine biology, sedimentology and archaeology (Fig. 6f). Another DOM application can be found in tethered robot umbilical modeling and control. Negative buoyancy cable can be modeled in real time as a simple catenary shape and tracked to control a tethered ROV [80].
海洋机器人技术：水下抓取技术几十年来一直由石油和天然气行业主导，导致用于检查和维护任务的机器配备强力抓取器（图 6e）。逐渐地，需求转向了海洋生物学、沉积学和考古学中更细致的任务（图 6f）。另一个 DOM 应用可以在系留机器人脐带建模和控制中找到。负浮力电缆可以实时建模为简单的悬链形状，并用于跟踪控制系留 ROV [80]。

The revolution of robots from automating repetitive tasks to humanizing robot behaviours is taking place with better hardware, robust sensing capabilities, accurate modeling, increasingly versatile planning and advanced control. Manipulation of deformable objects breaks fundamental assumptions in robotics such as rigidity, known dynamics models and low dimensional state space. It therefore requires breakthroughs in all the areas mentioned above, and serves as a great testbench for novel ideas in both robotic hardware and software. A summary of challenges and ideas discussed are presented in Fig. 7.
机器人从自动化重复任务到人性化行为的革命正在通过更好的硬件、强大的感知能力、精确的建模、日益多样化的规划和先进的控制来实现。对可变形物体的操作打破了机器人学中的基本假设，如刚性、已知的动力学模型和低维状态空间。因此，它需要在上述所有领域取得突破，并为机器人硬件和软件中的新想法提供了一个极好的测试平台。图 7 中总结了讨论的挑战和想法。

In terms of hardware, recently, the community has been shifting more and more from rigid to soft robots. Robotic manipulation is also gradually shifting from rigid to deformable objects. One open question is if some of the algorithms in one
在硬件方面，最近社区越来越多地从刚性机器人转向软体机器人。机器人操作也逐渐从刚性物体转向可变形物体。一个开放的问题是，某些算法是否可以从一个

Fig. 6. Various applications of DOM - (a): laundry-folding robot from Seven Dreamers Laboratories Inc. [81], (b): A mock-up for robotics dressing assistance, ©: a robot picking a flexible bag of goods on the shelf, courtesy AIRLab Delft [82], (d): autonomous surgical manipulation by the dVRK system [83], (e): ROV Victor 6000 sampling black smokers (IFREMER/GENAVIR) courtesy D. Desbruyères, (f): Ultra soft underwater gripper for jellyfish [84]
图 6. DOM 的各种应用 - (a): 来自 Seven Dreamers Laboratories Inc.的叠衣机器人[81], (b): 机器人穿衣辅助的模型, ©: 机器人在货架上拾取柔性商品袋, 由 AIRLab Delft 提供[82], (d): 由 dVRK 系统进行的自主手术操作[83], (e): ROV Victor 6000 采集黑烟囱样本(IFREMER/GENAVIR), 由 D. Desbruyères 提供, (f): 用于水母的超软水下夹持器[84]

Fig. 7. A summary of open research problems and ideas/methods to pursuit discussed in this paper. Research problems in each subarea are written with bold black texts, whereas ideas/methods to resolve them are marked with bold white texts in red eclipses.
图 7. 本文讨论的开放研究问题及追求思路/方法的总结。各子领域的研究问题以黑色粗体文本标注，而解决这些问题的思路/方法则在红色椭圆中以白色粗体文本标记。
field are transferable to the other? We believe the interaction between a soft robot and a deformable object will bring more challenges to the robotic community.
软机器人与可变形物体之间的互动将为机器人领域带来更多挑战。

Sensing plays a vital part in robotics manipulation of deformable objects. Depending on the nature and complexity of the task, one or multiple fused sensing modes may be needed. Machine learning will facilitate the development of robust algorithms to process data from different sensors, to generate meaningful representations of deformation.
感知在机器人对可变形物体的操作中起着至关重要的作用。根据任务的性质和复杂性，可能需要一种或多种融合的感知模式。机器学习将促进开发出能够处理来自不同传感器的数据，并生成有意义变形表示的鲁棒算法。
All models are wrong, some are useful. We do not believe there exists the “best” model for deformation. While more and more models tend to be data-driven, we would like to draw the readers’ attention to the importance of physical models for studying interactions.
所有模型都是错误的，但有些是有用的。我们不相信存在“最佳”的变形模型。尽管越来越多的模型倾向于数据驱动，但我们想提醒读者注意物理模型在研究相互作用中的重要性。

For planning, current research lacks a high level semantic reasoning of the DOM task. Furthermore, while often the purpose of planning is to avoid contact and collision, we argue that for DOM, it can be very useful to plan for contact.
在规划方面，当前的研究缺乏对 DOM 任务的高层次语义推理。此外，虽然规划的目的通常是避免接触和碰撞，但我们认为对于 DOM 来说，规划接触可能非常有用。

Under-actuation is a key challenge of DOM, due to the deformable bodies’ high DoF. Another practical issue introduced with deformation is contact loss during manipulation; future controllers should be able to detect contact loss and to react accordingly.
欠驱动是 DOM 的一个关键挑战，源于可变形体的高自由度。变形带来的另一个实际问题是操作过程中的接触丢失；未来的控制器应能检测接触丢失并作出相应反应。

[1] “La fontaine memory care.” [Online]. Available: https://www. lafontaine-mc.com/
[1] “La fontaine memory care.” [在线]。可访问：https://www. lafontaine-mc.com/
[2] “Cable harness.” [Online]. Available: https://www.nai-group.com/ products/custom-cable-harness/
[2] “线束。” [在线]。可用：https://www.nai-group.com/products/custom-cable-harness/
[3] B. G. Soe, “A woman harvesting seasonal fruit in a garden,” May 2020. [Online]. Available: https://commons.wikimedia.org/wiki/File: A_woman_harvesting_seasonal_fruit_in_a_garden_(May_2020).jpg
[3] B. G. Soe, “一位女士在花园里采摘季节性水果，” 2020 年 5 月. [在线]. 可获取: https://commons.wikimedia.org/wiki/File: A_woman_harvesting_seasonal_fruit_in_a_garden_(May_2020).jpg
[4] J. C. Mutter, “Plastic surgeon Vishal Kapoor, MD performing abdominoplasty surgery.” Oct 2010. [Online]. Available: https://commons. wikimedia.org/wiki/File:Vishal_Kapoor_MD_TummyTuck_Suture.jpg
[4] J. C. Mutter, “整形外科医生 Vishal Kapoor, MD 正在进行腹部整形手术。” 2010 年 10 月。[在线]。可访问：https://commons.wikimedia.org/wiki/File:Vishal_Kapoor_MD_TummyTuck_Suture.jpg
[5] J. Zhu, B. Navarro, R. Passama, P. Fraisse, A. Crosnier, and A. Cherubini, “Robotic manipulation planning for shaping deformable linear objects with environmental contacts,” IEEE Robotics and Automation Letters, vol. 5, no. 1, pp. 16-23, 2019.
[5] J. Zhu, B. Navarro, R. Passama, P. Fraisse, A. Crosnier, 和 A. Cherubini, “利用环境接触进行可变形线性物体成形的机器人操作规划,” IEEE Robotics and Automation Letters, vol. 5, no. 1, pp. 16-23, 2019.
[6] P. Jiménez, “Survey on model-based manipulation planning of deformable objects,” Robotics and Computer-integrated Manufacturing, vol. 28, no. 2, pp. 154-163, 2012.
[6] P. Jiménez, “基于模型的变形物体操作规划综述,” Robotics and Computer-integrated Manufacturing, vol. 28, no. 2, pp. 154-163, 2012.
[7] R. Herguedas, G. López-Nicolás, R. Aragüés, and C. Sagüés, “Survey on multi-robot manipulation of deformable objects,” in 2019 24th IEEE International Conference on Emerging Technologies and Factory Automation (ETFA). IEEE, 2019, pp. 977-984.
[7] R. Herguedas, G. López-Nicolás, R. Aragüés, 和 C. Sagüés, “关于多机器人操纵可变形物体的调查,” 发表于 2019 年第 24 届 IEEE 国际新兴技术与工厂自动化会议（ETFA）。IEEE, 2019, 第 977-984 页。
[8] F. Nadon, A. J. Valencia, and P. Payeur, “Multi-modal sensing and robotic manipulation of non-rigid objects: A survey,” Robotics, vol. 7, no. 4, p. 74, 2018.
[8] F. Nadon, A. J. Valencia, 和 P. Payeur, “多模态传感与非刚性物体的机器人操作：综述,” Robotics, 卷 7, 编号 4, 第 74 页, 2018.
[9] V. E. Arriola-Rios, P. Guler, F. Ficuciello, D. Kragic, B. Siciliano, and J. L. Wyatt, “Modeling of deformable objects for robotic manipulation: A tutorial and review,” Frontiers in Robotics and AI, vol. 7, 2020.
[9] V. E. Arriola-Rios, P. Guler, F. Ficuciello, D. Kragic, B. Siciliano, 和 J. L. Wyatt, “机器人操作中可变形物体的建模：教程与综述,” Frontiers in Robotics and AI, 卷 7, 2020.
[10] J. Sanchez, J.-A. Corrales, B.-C. Bouzgarrou, and Y. Mezouar, “Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey,” Int. J. of Robotics Research, 2018.
[10] J. Sanchez, J.-A. Corrales, B.-C. Bouzgarrou, 和 Y. Mezouar, “家庭和工业应用中可变形物体的机器人操作与感知：综述,” 《国际机器人研究杂志》, 2018.
[11] H. Yin, A. Varava, and D. Kragic, “Modeling, learning, perception, and control methods for deformable object manipulation,” Science Robotics, vol. 6, no. 54, 2021.
[11] H. Yin, A. Varava, 和 D. Kragic, “用于可变形物体操作的建模、学习、感知和控制方法,” Science Robotics, 卷 6, 期 54, 2021.
[12] Z. Hu, T. Han, P. Sun, J. Pan, and D. Manocha, “3-d deformable object manipulation using deep neural networks,” IEEE Robotics and Automation Letters, vol. 4, no. 4, pp. 4255-4261, 2019.
[12] Z. Hu, T. Han, P. Sun, J. Pan, 和 D. Manocha, “使用深度神经网络进行三维可变形物体操作,” IEEE Robotics and Automation Letters, 卷 4, 期 4, 页 4255-4261, 2019.
[13] A. Cherubini, V. Ortenzi, A. Cosgun, R. Lee, and P. Corke, “Modelfree vision-based shaping of deformable plastic materials,” The Int. J. of Robotics Research, 2020.
[13] A. Cherubini, V. Ortenzi, A. Cosgun, R. Lee, 和 P. Corke, “基于无模型视觉的可变形塑料材料成型,” 《国际机器人研究杂志》, 2020.
[14] H. Liu, M. Selvaggio, P. Ferrentino, R. Moccia, S. Pirozzi, U. Bracale, and F. Ficuciello, “The musha hand ii: A multi-functional hand for robot-assisted laparoscopic surgery,” IEEE/ASME Transactions on Mechatronics, 2020.
[14] H. Liu, M. Selvaggio, P. Ferrentino, R. Moccia, S. Pirozzi, U. Bracale, 和 F. Ficuciello, “Musha 手 II：用于机器人辅助腹腔镜手术的多功能手，” IEEE/ASME Transactions on Mechatronics, 2020.
[15] C. Della Santina, G. Grioli, M. Catalano, A. Brando, and A. Bicchi, “Dexterity augmentation on a synergistic hand: the pisa/iit softhand+,” in 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids). IEEE, 2015, pp. 497-503.
[15] C. Della Santina, G. Grioli, M. Catalano, A. Brando, 和 A. Bicchi, “Dexterity augmentation on a synergistic hand: the pisa/iit softhand+,” 在 2015 IEEE-RAS 第 15 届国际仿人机器人会议 (Humanoids) 中。IEEE, 2015, 第 497-503 页。
[16] S. Abondance, C. B. Teeple, and R. J. Wood, “A dexterous soft robotic hand for delicate in-hand manipulation,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 5502-5509, 2020.
[16] S. Abondance, C. B. Teeple, 和 R. J. Wood, “一种用于精细手内操作的灵巧软机器人手,” IEEE Robotics and Automation Letters, 卷 5, 期 4, 页 5502-5509, 2020.
[17] F. Alambeigi, Z. Wang, R. Hegeman, Y.-H. Liu, and M. Armand, “Autonomous data-driven manipulation of unknown anisotropic deformable tissues using unmodelled continuum manipulators,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 254-261, 2018.
[17] F. Alambeigi, Z. Wang, R. Hegeman, Y.-H. Liu, 和 M. Armand, “使用未建模的连续体机械手对未知各向异性可变形组织进行自主数据驱动操作,” IEEE Robotics and Automation Letters, 卷 4, 期 2, 页 254-261, 2018.
[18] Y. Hao and Y. Visell, “Beyond soft hands: Efficient grasping with non-anthropomorphic soft grippers,” Frontiers in Robotics and AI, vol. 8, p. 210, 2021. [Online]. Available: https://www.frontiersin.org/ article/10.3389/frobt. 2021.632006
[18] Y. Hao 和 Y. Visell，“超越软手：非拟人化软夹爪的高效抓取”，《机器人学与人工智能前沿》，第 8 卷，第 210 页，2021 年。[在线]。可访问：https://www.frontiersin.org/ article/10.3389/frobt. 2021.632006
[19] “This fruit-picking robot can pick up to 25,000 raspberries a day, and it could someday replace human workers.” [Online]. Available: https://www.businessinsider.nl/ robot-picks-25000-raspberries-a-day-outpaces-human-workers-2019-5/ ?jwsource=cl
[19] “这款水果采摘机器人每天可以采摘多达 25,000 颗树莓，未来可能取代人工。” [在线]。可用：https://www.businessinsider.nl/ robot-picks-25000-raspberries-a-day-outpaces-human-workers-2019-5/ ?jwsource=cl
[20] D. M. Vogt, K. P. Becker, B. T. Phillips, M. A. Graule, R. D. Rotjan, T. M. Shank, E. E. Cordes, R. J. Wood, and D. F. Gruber, “Shipboard design and fabrication of custom 3d-printed soft robotic manipulators for the investigation of delicate deep-sea organisms,” PLOS ONE, vol. 13, no. 8, pp. 1-16, 082018.
[20] D. M. Vogt, K. P. Becker, B. T. Phillips, M. A. Graule, R. D. Rotjan, T. M. Shank, E. E. Cordes, R. J. Wood, 和 D. F. Gruber, “船舶上设计和制造定制的 3D 打印软体机器人操纵器用于研究脆弱的深海生物,” PLOS ONE, 卷 13, 期 8, 页 1-16, 082018.
[21] F. Ficuciello, A. Migliozzi, E. Coevoet, A. Petit, and C. Duriez, “Fembased deformation control for dexterous manipulation of 3 d soft objects,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2018, pp. 4007-4013.
[21] F. Ficuciello, A. Migliozzi, E. Coevoet, A. Petit, 和 C. Duriez, “基于有限元的形变控制用于 3D 软物体的灵巧操作,” 在 2018 年 IEEE/RSJ 智能机器人与系统国际会议(IROS)上, 2018, 第 4007-4013 页。
[22] E. Coevoet, A. Escande, and C. Duriez, “Optimization-based inverse model of soft robots with contact handling,” IEEE Robotics and Automation Letters, vol. 2, no. 3, pp. 1413-1419, 2017.
[22] E. Coevoet, A. Escande, 和 C. Duriez, “基于优化的软体机器人逆模型及其接触处理,” IEEE Robotics and Automation Letters, vol. 2, no. 3, pp. 1413-1419, 2017.
[23] J. Luo, E. Solowjow, C. Wen, J. A. Ojea, and A. M. Agogino, “Deep reinforcement learning for robotic assembly of mixed deformable and rigid objects,” in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 2062-2069.
[23] J. Luo, E. Solowjow, C. Wen, J. A. Ojea, 和 A. M. Agogino, “深度强化学习在混合可变形和刚性物体机器人装配中的应用,” 发表于 2018 IEEE/RSJ 智能机器人与系统国际会议 (IROS). IEEE, 2018, 第 2062-2069 页.
[24] Y. Hayami, P. Shi, W. Wan, I. G. Ramirez-Alpizar, and K. Harada, “Multi-dimensional error identification during robotic snap assembly,” in IFToMM World Congress on Mechanism and Machine Science. Springer, 2019, pp. 2189-2198.
[24] Y. Hayami, P. Shi, W. Wan, I. G. Ramirez-Alpizar, 和 K. Harada, “机器人在快速装配过程中的多维错误识别,” 发表于 IFToMM 世界机械与机器科学大会. Springer, 2019, 第 2189-2198 页.
[25] A. Nair, D. Chen, P. Agrawal, P. Isola, P. Abbeel, J. Malik, and S. Levine, “Combining self-supervised learning and imitation for vision-based rope manipulation,” in 2017 IEEE international conference on robotics and automation (ICRA). IEEE, 2017, pp. 2146-2153.
[25] A. Nair, D. Chen, P. Agrawal, P. Isola, P. Abbeel, J. Malik, 和 S. Levine, “结合自监督学习和模仿进行基于视觉的绳索操作，” 2017 年 IEEE 国际机器人与自动化会议（ICRA）。IEEE, 2017, 第 2146-2153 页。
[26] M. Yan, Y. Zhu, N. Jin, and J. Bohg, “Self-supervised learning of state estimation for manipulating deformable linear objects,” IEEE robotics and automation letters, vol. 5, no. 2, pp. 2372-2379, 2020
[26] M. Yan, Y. Zhu, N. Jin, 和 J. Bohg, “用于操纵可变形线性物体的状态估计的自监督学习,” IEEE robotics and automation letters, vol. 5, no. 2, pp. 2372-2379, 2020
[27] L. Sun, G. Aragon-Camarasa, S. Rogers, and J. P. Siebert, “Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening,” in 2015 IEEE international conference on robotics and automation (ICRA). IEEE, 2015, pp. 185-192.
[27] L. Sun, G. Aragon-Camarasa, S. Rogers, 和 J. P. Siebert, “使用主动立体机器人头进行精确的服装表面分析及其在双臂展平中的应用，” 2015 年 IEEE 国际机器人与自动化会议（ICRA）。IEEE, 2015, 第 185-192 页。
[28] Y. Li, D. Xu, Y. Yue, Y. Wang, S.-F. Chang, E. Grinspun, and P. K. Allen, “Regrasping and unfolding of garments using predictive thin shell modeling,” in 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2015, pp. 1382-1388.
[28] Y. Li, D. Xu, Y. Yue, Y. Wang, S.-F. Chang, E. Grinspun, 和 P. K. Allen, “使用预测性薄壳模型进行衣物的重新抓取和展开,” 在 2015 年 IEEE 国际机器人与自动化会议 (ICRA). IEEE, 2015, 第 1382-1388 页.
29] C. Cheng and D. Berenson, “Occlusion-robust deformable object tracking without physics simulation,” in IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2019.
29] C. Cheng 和 D. Berenson, “无需物理模拟的遮挡鲁棒性可变形物体跟踪,” 在 IEEE/RSJ 国际智能机器人与系统会议 (IROS) 上, 2019.
[30] A. Petit, V. Lippiello, G. A. Fontanelli, and B. Siciliano, “Tracking elastic deformable objects with an rgb-d sensor for a pizza chef robot,” Robotics and Autonomous Systems, vol. 88, pp. 187-201, 2017
[30] A. Petit, V. Lippiello, G. A. Fontanelli, 和 B. Siciliano, “使用 RGB-D 传感器跟踪弹性变形物体以实现披萨厨师机器人,” Robotics and Autonomous Systems, 卷 88, 页 187-201, 2017
[31] A. J. Valencia, F. Nadon, and P. Payeur, “Toward real-time 3d shape tracking of deformable objects for robotic manipulation and shape control,” in 2019 IEEE SENSORS. IEEE, 2019, pp. 1-4
[31] A. J. Valencia, F. Nadon, 和 P. Payeur, “Toward real-time 3d shape tracking of deformable objects for robotic manipulation and shape control,” 在 2019 IEEE SENSORS 中。IEEE, 2019, 第 1-4 页
[32] G. Rouhafzay, A.-M. Cretu, and P. Payeur, “Transfer of learning from vision to touch: A hybrid deep convolutional neural network for visuotactile 3d object recognition,” Sensors, vol. 21, no. 1, p. 113, 2021.
[32] G. Rouhafzay, A.-M. Cretu, 和 P. Payeur, “从视觉到触觉的学习迁移：一种用于视觉触觉三维物体识别的混合深度卷积神经网络,” Sensors, 卷 21, 期 1, 页 113, 2021.
[33] W. Yuan, M. A. Srinivasan, and E. H. Adelson, “Estimating object hardness with a gelsight touch sensor,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2016, pp. 208-215.
[33] W. Yuan, M. A. Srinivasan, 和 E. H. Adelson, “使用 GelSight 触觉传感器估计物体硬度,” 发表于 2016 年 IEEE/RSJ 智能机器人与系统国际会议 (IROS). IEEE, 2016, 第 208-215 页.
[34] W. Yuan, S. Dong, and E. H. Adelson, “Gelsight: High-resolution robot tactile sensors for estimating geometry and force,” Sensors, vol. 17, no. 12, p. 2762, 2017.
[34] W. Yuan, S. Dong, 和 E. H. Adelson, “Gelsight: 用于估计几何形状和力的高分辨率机器人触觉传感器,” Sensors, 卷 17, 期 12, 页 2762, 2017.
[35] P. Güler, Y. Bekiroglu, X. Gratal, K. Pauwels, and D. Kragic, “What’s in the container? classifying object contents from vision and touch,” in 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE, 2014, pp. 3961-3968.
[35] P. Güler, Y. Bekiroglu, X. Gratal, K. Pauwels, 和 D. Kragic, “容器里有什么？从视觉和触觉对物体内容进行分类,” 在 2014 IEEE/RSJ 智能机器人与系统国际会议. IEEE, 2014, pp. 3961-3968.
[36] W. Yuan, Y. Mo, S. Wang, and E. H. Adelson, “Active clothing material perception using tactile sensing and deep learning,” in 2018 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2018, pp. 4842-4849.
[36] W. Yuan, Y. Mo, S. Wang, 和 E. H. Adelson, “利用触觉感知和深度学习进行主动服装材料感知,” 在 2018 IEEE 国际机器人与自动化会议 (ICRA). IEEE, 2018, 第 4842-4849 页.
[37] Y. She, S. Wang, S. Dong, N. Sunil, A. Rodriguez, and E. Adelson, “Cable manipulation with a tactile-reactive gripper,” in Robotics: Science and Systems, 2020.
[37] Y. She, S. Wang, S. Dong, N. Sunil, A. Rodriguez, 和 E. Adelson, “使用触觉反应夹持器进行电缆操作,” 发表于 Robotics: Science and Systems, 2020.
[38] S. Levine, C. Finn, T. Darrell, and P. Abbeel, “End-to-end training of deep visuomotor policies,” J. of Machine Learning Research, vol. 17, no. 39, pp. 1-40, 2016.
[38] S. Levine, C. Finn, T. Darrell, 和 P. Abbeel, “深度视觉运动策略的端到端训练,” 《机器学习研究杂志》, 卷 17, 期 39, 页 1-40, 2016.
[39] J. Matas, S. James, and A. J. Davison, “Sim-to-real reinforcement learning for deformable object manipulation,” in Conference on Robot Learning. PMLR, 2018, pp. 734-743.
[39] J. Matas, S. James, 和 A. J. Davison, “用于可变形物体操作的模拟到现实强化学习,” 在机器人学习会议上. PMLR, 2018, 第 734-743 页.
[40] J. Zhu, D. Navarro-Alarcon, R. Passama, and A. Cherubini, “Visionbased manipulation of deformable and rigid objects using subspace projections of 2d contours,” Robotics and Autonomous Systems, vol. 142, 2021.
[40] J. Zhu, D. Navarro-Alarcon, R. Passama, 和 A. Cherubini, “基于视觉的变形和刚性物体操作使用二维轮廓的子空间投影,” Robotics and Autonomous Systems, 卷 142, 2021.
[41] D. Navarro-Alarcon and Y.-H. Liu, “Fourier-based shape servoing: A new feedback method to actively deform soft objects into desired 2-D image contours,” IEEE Trans. on Robotics, vol. 34, no. 1, pp. 272-1279, 2018.
[41] D. Navarro-Alarcon 和 Y.-H. Liu, “基于傅里叶的形状伺服：一种将软物体主动变形为所需二维图像轮廓的新反馈方法,” IEEE Trans. on Robotics, vol. 34, no. 1, pp. 272-1279, 2018.
[42] R. Lagneau, A. Krupa, and M. Marchal, “Automatic shape control of deformable wires based on model-free visual servoing,” IEEE Robotics and Automation Letters, vol. 5, no. 4, pp. 5252-5259, 2020.
[42] R. Lagneau, A. Krupa, 和 M. Marchal, “基于无模型视觉伺服的可变形线自动形状控制,” IEEE Robotics and Automation Letters, 卷 5, 期 4, 页 5252-5259, 2020.
[43] D. McConachie and D. Berenson, “Bandit-Based Model Selection for Deformable Object Manipulation,” Workshop on the Algorithmic Foundations of Robotics (WAFR), 2016.
[43] D. McConachie 和 D. Berenson, “Bandit-Based Model Selection for Deformable Object Manipulation,” Workshop on the Algorithmic Foundations of Robotics (WAFR), 2016.
[44] E. Yoshida, K. Ayusawa, I. G. Ramirez-Alpizar, K. Harada, and C. Duriez, “Simulation-based optimal motion planning for deformable object,” in 2015 IEEE International Workshop in Advanced Robotics and its Social Impacts (ARSO), 2015.
[44] E. Yoshida, K. Ayusawa, I. G. Ramirez-Alpizar, K. Harada, 和 C. Duriez, “基于仿真的可变形物体最优运动规划,” 发表于 2015 年 IEEE 先进机器人及其社会影响国际研讨会 (ARSO), 2015.
[45] A. J. Valencia and P. Payeur, “Combining self-organizing and graph neural networks for modeling deformable objects in robotic manipulation,” Frontiers in Robotics and AI, vol. 7, 2020.
[45] A. J. Valencia 和 P. Payeur, “结合自组织和图神经网络用于机器人操作中可变形物体的建模,” Frontiers in Robotics and AI, vol. 7, 2020.
[46] P. W. Battaglia, R. Pascanu, M. Lai, D. Rezende, and K. Kavukcuoglu, “Interaction networks for learning about objects, relations and physics,” 2016.
[46] P. W. Battaglia, R. Pascanu, M. Lai, D. Rezende, 和 K. Kavukcuoglu, “用于学习物体、关系和物理的交互网络,” 2016.

47] D. M ${}^{\text{c Conachie, T. Power, P. Mitrano, and D. Berenson, "Learning When}}$${}^{\text{c Conachie, T. Power, P. Mitrano, and D. Berenson, "Learning When }}$^("c Conachie, T. Power, P. Mitrano, and D. Berenson, "Learning When "){ }^{\text {c Conachie, T. Power, P. Mitrano, and D. Berenson, "Learning When }} to Trust a Dynamics Model for Planning in Reduced State Spaces," IEEE Robotics and Automation Letters (RA-L), 2020.
47] D. M ${}^{\text{c Conachie, T. Power, P. Mitrano, and D. Berenson, "Learning When}}$${}^{\text{c Conachie, T. Power, P. Mitrano, and D. Berenson, "Learning When }}$^("c Conachie, T. Power, P. Mitrano, and D. Berenson, "Learning When "){ }^{\text {c Conachie, T. Power, P. Mitrano, and D. Berenson, "Learning When }} 在减少状态空间中信任动力学模型进行规划，《IEEE 机器人与自动化快报》（RA-L），2020 年。
[48] T. Power and D. Berenson, “Keep it Simple: Data-efficient Learning for Controlling Complex Systems with Simple Models,” IEEE Robotics and Automation Letters (RA-L), 2021.
[48] T. Power 和 D. Berenson, “保持简单：使用简单模型控制复杂系统的数据高效学习,” IEEE Robotics and Automation Letters (RA-L), 2021.
[49] P. Mitrano, D. McConachie, and D. Berenson, “Learning where to trust unreliable models in an unstructured world for deformable object manipulation,” Science Robotics, vol. 6, no. 54, p. eabd8170, 2021.
[49] P. Mitrano, D. McConachie, 和 D. Berenson, “在非结构化世界中学习何处信任不可靠模型以进行可变形物体操作,” Science Robotics, 卷 6, 期 54, 页 eabd8170, 2021.
[50] X. Lin, Y. Wang, J. Olkin, and D. Held, “Softgym: Benchmarking deep reinforcement learning for deformable object manipulation,” arXiv preprint arXiv:2011.07215, 2020.
[50] X. Lin, Y. Wang, J. Olkin, 和 D. Held, “Softgym: 用于可变形物体操作的深度强化学习基准测试,” arXiv 预印本 arXiv:2011.07215, 2020.
[51] E. Coevoet, T. Morales-Bieze, F. Largilliere, Z. Zhang, M. Thieffry, M. Sanz-Lopez, B. Carrez, D. Marchal, O. Goury, J. Dequidt et al., “Software toolkit for modeling, simulation, and control of soft robots,” Advanced Robotics, vol. 31, no. 22, pp. 1208-1224, 2017.
[51] E. Coevoet, T. Morales-Bieze, F. Largilliere, Z. Zhang, M. Thieffry, M. Sanz-Lopez, B. Carrez, D. Marchal, O. Goury, J. Dequidt 等，“用于软机器人建模、仿真和控制的软件工具包”，《高级机器人学》，第 31 卷，第 22 期，第 1208-1224 页，2017 年。
[52] Y. Hu, J. Liu, A. Spielberg, J. B. Tenenbaum, W. T. Freeman, J. Wu, D. Rus, and W. Matusik, “Chainqueen: A real-time differentiable physical simulator for soft robotics,” in 2019 International conference on robotics and automation (ICRA). IEEE, 2019, pp. 6265-6271
[52] Y. Hu, J. Liu, A. Spielberg, J. B. Tenenbaum, W. T. Freeman, J. Wu, D. Rus, 和 W. Matusik, “Chainqueen: 一种用于软体机器人的实时可微分物理模拟器,” 在 2019 年国际机器人与自动化会议 (ICRA) 上. IEEE, 2019, 第 6265-6271 页
[53] T. Li, K. Srinivasan, M. Q.-H. Meng, W. Yuan, and J. Bohg, “Learning hierarchical control for robust in-hand manipulation,” in 2020 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2020, pp. 8855-8862.
[53] T. Li, K. Srinivasan, M. Q.-H. Meng, W. Yuan, 和 J. Bohg, “学习分层控制以实现稳健的手内操作,” 发表于 2020 IEEE 国际机器人与自动化会议 (ICRA). IEEE, 2020, 第 8855-8862 页.
[54] D. McConachie, A. Dobson, M. Ruan, and D. Berenson, “Manipulating deformable objects by interleaving prediction, planning, and control,” arXiv preprint arXiv:2001.09950, 2020.
[54] D. McConachie, A. Dobson, M. Ruan, 和 D. Berenson, “通过交错预测、规划和控制来操纵可变形物体,” arXiv 预印本 arXiv:2001.09950, 2020.
[55] I. Ramirez-Alpizar, K. Harada, and E. Yoshida, “Motion planning for dual-arm assembly of ring-shaped elastic objects,” in IEEE-RAS International Conference on Humanoid Robots, 2014, pp. 594-600.
[55] I. Ramirez-Alpizar, K. Harada, 和 E. Yoshida, “用于环状弹性物体双臂装配的运动规划,” 发表于 IEEE-RAS 国际人形机器人会议, 2014, 第 594-600 页。
[56] J. Alonso-Mora, R. Knepper, R. Siegwart, and D. Rus, “Local motion planning for collaborative multi-robot manipulation of deformable objects,” in 2015 IEEE international conference on robotics and automation (ICRA). IEEE, 2015, pp. 5495-5502.
[56] J. Alonso-Mora, R. Knepper, R. Siegwart, 和 D. Rus, “用于协作多机器人变形物体操作的局部运动规划,” 发表于 2015 年 IEEE 国际机器人与自动化会议 (ICRA). IEEE, 2015, 第 5495-5502 页.
[57] P. Zhou, J. Zhu, S. Huo, and D. Navarro-Alarcon, “LaSeSOM: A latent and semantic representation framework for soft object manipulation,” IEEE Robotics and Automation Letters, 2021.
[57] P. Zhou, J. Zhu, S. Huo, 和 D. Navarro-Alarcon, “LaSeSOM: 一种用于软物体操作的潜在和语义表示框架,” IEEE Robotics and Automation Letters, 2021.
[58] A. X. Lee, H. Lu, A. Gupta, S. Levine, and P. Abbeel, “Learning force-based manipulation of deformable objects from multiple demonstrations,” in 2015 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2015, pp. 177-184.
[58] A. X. Lee, H. Lu, A. Gupta, S. Levine, 和 P. Abbeel, “从多次演示中学习基于力的可变形物体操作,” 2015 年 IEEE 国际机器人与自动化会议(ICRA). IEEE, 2015, 第 177-184 页.
[59] Y. Wu, W. Yan, T. Kurutach, L. Pinto, and P. Abbeel, “Learning to manipulate deformable objects without demonstrations,” arXiv preprint arXiv:1910.13439, 2019.
[59] Y. Wu, W. Yan, T. Kurutach, L. Pinto, 和 P. Abbeel, “无需演示学习操纵可变形物体,” arXiv 预印本 arXiv:1910.13439, 2019.
[60] J. Van Den Berg, S. Miller, K. Goldberg, and P. Abbeel, “Gravitybased robotic cloth folding,” in Algorithmic Foundations of Robotics IX. Springer, 2010, pp. 409-424.
[60] J. Van Den Berg, S. Miller, K. Goldberg, 和 P. Abbeel, “基于重力的机器人布料折叠,” 收录于《机器人学的算法基础 IX》. Springer, 2010, 第 409-424 页.
[61] I. Mitsioni, Y. Karayiannidis, J. A. Stork, and D. Kragic, “Data-driven model predictive control for the contact-rich task of food cutting,” in 2019 IEEE-RAS 19th International Conference on Humanoid Robots (Humanoids), 2019, pp. 244-250.
[61] I. Mitsioni, Y. Karayiannidis, J. A. Stork, 和 D. Kragic, “基于数据驱动的模型预测控制在食物切割这一接触丰富任务中的应用,” 发表于 2019 IEEE-RAS 第 19 届国际人形机器人会议 (Humanoids), 2019, 第 244-250 页。
[62] H. Su, C. Yang, G. Ferrigno, and E. De Momi, “Improved humanrobot collaborative control of redundant robot for teleoperated minimally invasive surgery,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1447-1453, 2019.
[62] H. Su, C. Yang, G. Ferrigno, 和 E. De Momi, “改进的冗余机器人人机协作控制用于远程微创手术,” IEEE Robotics and Automation Letters, vol. 4, no. 2, pp. 1447-1453, 2019.
$63]$$63]$63]63] M. Aranda, J. A. C. Ramon, Y. Mezouar, A. Bartoli, and E. Özgür, “Monocular visual shape tracking and servoing for isometrically deforming objects,” in 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS). IEEE, 2020, pp. 7542-7549.
$63]$$63]$63]63] M. Aranda, J. A. C. Ramon, Y. Mezouar, A. Bartoli, 和 E. Özgür, “单目视觉形状跟踪与伺服用于等距变形物体,” 发表于 2020 IEEE/RSJ 国际智能机器人与系统会议 (IROS). IEEE, 2020, 第 7542-7549 页.
[64] X. Li, X. Su, and Y.-H. Liu, “Vision-based robotic manipulation of flexible pcbs,” IEEE/ASME Transactions on Mechatronics, vol. 23, no. 6, pp. 2739-2749, 2018.
[64] X. Li, X. Su, 和 Y.-H. Liu, “基于视觉的柔性 PCB 机器人操作,” IEEE/ASME Transactions on Mechatronics, vol. 23, no. 6, pp. 2739-2749, 2018.
[65] X. Li, X. Su, Y. Gao, and Y.-H. Liu, “Vision-based robotic grasping and manipulation of usb wires,” in IEEE International Conference on Robotics and Automation, 2018, pp. 3482-3487.
[65] X. Li, X. Su, Y. Gao, 和 Y.-H. Liu, “基于视觉的机器人抓取与操作 USB 线缆,” 发表于 IEEE 国际机器人与自动化会议, 2018, 第 3482-3487 页。
[66] F. Nadon and P. Payeur, “Grasp selection for in-hand robotic manipulation of non-rigid objects with shape control,” in 2020 IEEE International Systems Conference (SysCon). IEEE, 2020, pp. 1-8.
[66] F. Nadon 和 P. Payeur, “用于非刚性物体形状控制的机器人手内操作的抓取选择,” 在 2020 IEEE 国际系统会议 (SysCon). IEEE, 2020, 第 1-8 页.
[67] M. Ruan, D. Mc Conachie, and D. Berenson, “Accounting for directional rigidity and constraints in control for manipulation of deformable objects without physical simulation,” in IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS). IEEE, 2018, pp. 512-519.
[67] M. Ruan, D. Mc Conachie, 和 D. Berenson, “在不进行物理仿真的情况下，考虑方向刚性和约束的控制以操纵可变形物体，” 发表于 IEEE/RSJ 国际智能机器人与系统会议 (IROS). IEEE, 2018, 第 512-519 页。
[68] K. Nagata, “Laundry-phobics’ dreams crushed as tokyo-based developer of laundroid robot files for bankruptcy,” Apr 2019. [Online]. Available: https:
[68] K. Nagata, “洗衣恐惧症患者的梦想破灭，东京的 Laundroid 机器人开发商申请破产，” 2019 年 4 月。 [在线]. 可访问: https:
//www.japantimes.co.jp/news/2019/04/24/business/corporate-business/ tokyo-based-developer-laundry-folding-robot-files-bankruptcy/
//www.japantimes.co.jp/news/2019/04/24/business/corporate-business/ 东京开发商洗衣折叠机器人申请破产
[69] A. Verleysen, M. Biondina, and F. Wyffels, “Video dataset of human demonstrations of folding clothing for robotic folding,” The International Journal of Robotics Research, vol. 39, no. 9, pp. 1031-1036, 2020.
[69] A. Verleysen, M. Biondina, 和 F. Wyffels, “用于机器人折叠的人类折叠衣物演示视频数据集,” The International Journal of Robotics Research, 卷 39, 期 9, 页 1031-1036, 2020.
[70] S. Miller, J. Van Den Berg, M. Fritz, T. Darrell, K. Goldberg, and P. Abbeel, “A geometric approach to robotic laundry folding,” Int. J. of Robotics Research, vol. 31, no. 2, pp. 249-267, 2012.
[70] S. Miller, J. Van Den Berg, M. Fritz, T. Darrell, K. Goldberg, 和 P. Abbeel, “一种用于机器人折叠衣物的几何方法,” 《国际机器人研究杂志》, 卷 31, 期 2, 页 249-267, 2012.
[71] Y. Tsurumine, Y. Cui, E. Uchibe, and T. Matsubara, “Deep reinforcement learning with smooth policy update: Application to robotic cloth manipulation,” Robotics and Autonomous Systems, vol. 112, pp. 72-83, 2019.
[71] Y. Tsurumine, Y. Cui, E. Uchibe, 和 T. Matsubara, “深度强化学习与平滑策略更新：在机器人布料操作中的应用,” Robotics and Autonomous Systems, 卷 112, 页 72-83, 2019.
[72] J. Borràs, G. Alenyà, and C. Torras, “A grasping-centered analysis for cloth manipulation,” IEEE Transactions on Robotics, vol. 36, no. 3, pp. 924-936, 2020.
[72] J. Borràs, G. Alenyà, 和 C. Torras, “以抓取为中心的布料操作分析,” IEEE Transactions on Robotics, 卷 36, 期 3, 页 924-936, 2020.
[73] A. Clegg, W. Yu, J. Tan, C. K. Liu, and G. Turk, “Learning to dress: Synthesizing human dressing motion via deep reinforcement learning,” ACM Transactions on Graphics (TOG), vol. 37, no. 6, pp. 1-10, 2018.
[73] A. Clegg, W. Yu, J. Tan, C. K. Liu, 和 G. Turk, “学习穿衣：通过深度强化学习合成人类穿衣动作,” ACM Transactions on Graphics (TOG), vol. 37, no. 6, pp. 1-10, 2018.
[74] A. Clegg, Z. Erickson, P. Grady, G. Turk, C. C. Kemp, and C. K. Liu, “Learning to collaborate from simulation for robot-assisted dressing,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2746-2753, 2020.
[74] A. Clegg, Z. Erickson, P. Grady, G. Turk, C. C. Kemp, 和 C. K. Liu, “从模拟中学习协作以进行机器人辅助穿衣,” IEEE Robotics and Automation Letters, vol. 5, no. 2, pp. 2746-2753, 2020.
[75] R. P. Joshi, N. Koganti, and T. Shibata, “A framework for robotic clothing assistance by imitation learning,” Advanced Robotics, vol. 33, no. 22, pp. 1156-1174, 2019.
[75] R. P. Joshi, N. Koganti, 和 T. Shibata, “通过模仿学习实现机器人衣物辅助的框架,” 《高级机器人学》, 卷 33, 期 22, 页 1156-1174, 2019.
[76] G. Canal, G. Alenyà, and C. Torras, “Adapting robot task planning to user preferences: an assistive shoe dressing example,” Autonomous Robots, vol. 43, no. 6, pp. 1343-1356, 2019.
[76] G. Canal, G. Alenyà, 和 C. Torras, “Adapting robot task planning to user preferences: an assistive shoe dressing example,” Autonomous Robots, 卷 43, 期 6, 页 1343-1356, 2019.
[77] S. Li, N. Figueroa, A. Shah, and J. A. Shah, “Provably Safe and Efficient Motion Planning with Uncertain Human Dynamics,” in Proceedings of Robotics: Science and Systems, Virtual, July 2021.
[77] S. Li, N. Figueroa, A. Shah, 和 J. A. Shah, “具有不确定人类动态的可证明安全高效运动规划,” 发表于《机器人学：科学与系统》会议, 虚拟会议, 2021 年 7 月。
[78] T. Tamei, T. Matsubara, A. Rai, and T. Shibata, “Reinforcement learning of clothing assistance with a dual-arm robot,” in 2011 11th IEEE-RAS International Conference on Humanoid Robots. IEEE, 2011, pp. 733738.
[78] T. Tamei, T. Matsubara, A. Rai, 和 T. Shibata, “使用双臂机器人进行衣物辅助的强化学习,” 在 2011 年第 11 届 IEEE-RAS 国际人形机器人会议上. IEEE, 2011, 第 733-738 页.
[79] F. Zhang, A. Cully, and Y. Demiris, “Probabilistic real-time user posture tracking for personalized robot-assisted dressing,” IEEE Transactions on Robotics, vol. 35, no. 4, pp. 873-888, 2019.
[79] F. Zhang, A. Cully, 和 Y. Demiris, “个性化机器人辅助穿衣中的概率实时用户姿态跟踪,” IEEE Transactions on Robotics, 卷 35, 期 4, 页 873-888, 2019.
[80] M. Laranjeira, C. Dune, and V. Hugel, “Catenary-based visual servoing for tether shape control between underwater vehicles,” Ocean Engineering, vol. 200, p. 107018, 2020. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0029801820300949
[80] M. Laranjeira, C. Dune, 和 V. Hugel, “基于悬链线的视觉伺服用于水下航行器间缆索形状控制,” Ocean Engineering, 卷 200, 页 107018, 2020. [在线]. 可访问: https://www.sciencedirect.com/science/article/pii/S0029801820300949
[81] A. Wolf, “Japanese robotics biz helps lick laundry,” Dec 2017. [Online]. Available: https://www.twice.com/product/ japans-seven-dreamers-robot-solution-folding-laundry
[81] A. Wolf, “日本机器人业务助力洗衣，” 2017 年 12 月。 [在线]. 可获取: https://www.twice.com/product/japans-seven-dreamers-robot-solution-folding-laundry
[82] “The AI for retail (AIR) Lab Delft.” [Online]. Available: https: //icai.ai/airlab-delft/
[82] “零售人工智能（AIR）实验室代尔夫特。” [在线]. 可访问: https://icai.ai/airlab-delft/
[83] F. Alambeigi, Z. Wang, R. Hegeman, Y.-H. Liu, and M. Armand, “A robust data-driven approach for online learning and manipulation of unmodeled 3-d heterogeneous compliant objects,” IEEE Robotics and Automation Letters, vol. 3, no. 4, pp. 4140-4147, 2018.
[83] F. Alambeigi, Z. Wang, R. Hegeman, Y.-H. Liu, 和 M. Armand, “一种鲁棒的数据驱动方法用于在线学习和操作未建模的三维异质柔性物体,” IEEE Robotics and Automation Letters, 卷 3, 编号 4, 页码 4140-4147, 2018.
[84] N. R. Sinatra, C. B. Teeple, D. M. Vogt, K. K. Parker, D. F. Gruber, and R. J. Wood, “Ultragentle manipulation of delicate structures using a soft robotic gripper,” Science Robotics, vol. 4, no. 33, 2019.
[84] N. R. Sinatra, C. B. Teeple, D. M. Vogt, K. K. Parker, D. F. Gruber, 和 R. J. Wood, “使用软体机器人夹持器超轻柔地操作精细结构,” Science Robotics, 卷 4, 第 33 期, 2019.