Decomposition of a Dynamic Manipulation Task Robotic Sewing

bruce sewing machine
juki sewing machine price

IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 6, NO. 3, JUNE 1990 357
I
Parallel Process Decomposition of a Dynamic
Manipulation Task: Robotic Sewing
Abstmct- Conventional robot control schemes are inadequate for dynamic
tasks involving gross environmental uncertainty, and consequently,
current robots are restricted to artificial structured environments. The
task planner approach, as promoted by the AI community, is unsuited to
tasks involving interaction with a dynamic environment. An alternative
approach, which is based on concurrent control processes or behaviors,
can enable a system to perform dynamic tasks in an uncertain environment
with inherent error recovery and graceful degradation. Dynamic
tasks, such as locomotion and mobile robot navigation, have decomposed
into parallel processes, resulting in control systems that are simple
yet robust enough to operate in realistic unstructured environments.
This paper investigates parallel decomposition of manipulation tasks
that involve interaction with a dynamic environment. A methodology
for developing concurrent process controllers for such tasks is outlined.
A concurrent process control system was developed for a robotic sewing
cell that produced an edge seam on a fabric panel with arbitrary edge
curvature. Although a conventional sewing machine sewed up the fabric,
a robot arm manipulated the panel to modify its orientation and control
fabric tension. The sewing task was decomposed into four concurrent
processes within a superposition parallel architecture. Although each
process was based on simple linearized models, the complete system
demonstrated robustness to the unpredictable dynamic behavior of the
fabric panel.
I. INTRODUCTION
MO ST INDUSTRIAL robots are limited to simple un- constrained tasks in artificial structured environments.
To enable robots to perform skillful tasks in more realistic
environments, we require a systematic methodology for developing
intelligent control systems that incorporate multiple
dynamic sensors. Manipulation tasks, such as two-handed assembly
or operation of power tools, that can involve constrained
motion of the robot during interaction with a dynamic
environment are an important class of robot tasks that need
an intelligent control approach.
In classical control system design, a model of the task is
constructed, and a controller is then designed to achieve the
task objectives within specified performance criteria. Many
control schemes, such as adaptive control [15], [27] and
sliding-mode control [4 11, can accommodate bounded uncertainties
in the model of the controlled system. However, these
controllers are not generally robust to disturbances generated
by a dynamic interacting environment. Theoretical results on
the coupled stability of interacting dynamic systems together
with an impedance control design methodology for control-
Manuscript received October 27, 1988; revised January 26, 1990. This
work was supported by a TOMRB grant from the UK Department of Trade
and Industry and by a Sir Charles Clore post-doctoral fellowship.
The author is with the Faculty of Mechanical Engineering, Technion, Israel
Institute of Technology, Haifa, Israel.
IEEE Log Number 9035995.
ling such systems were presented in [ 113. This approach is
appropriate for single-objective tasks that do not require several
dynamic sensors but is inadequate for more complex tasks
and environments that have multiple objectives and require a
hierarchy of models.
The classical AI approach to the development of robots
for unstructured environments is based on the task planner
concept. Explicit models of the robot, the task, and the environment
are constructed, and sensors are used to update the
models and reduce model uncertainty. A robot-level program,
consisting of a sequence of motion commands and sensor measurements,
is then planned, and it should achieve the task objectives
for the perceived world [13], [17], [28], [30], [45].
This approach can easily handle several task objectives and
constraints since the planner searches for a solution through
a tree representing all possible permutations of robot-level
program sequences. Potentially, it could also handle unstructured
static environments if it were possible to represent all
relevant knowledge and develop a “common sense reasoning
capability. ” However, this is unlikely with current knowledge
representation and automated reasoning techniques [8], [20],
[43]. Task planning is essentially an off-line activity, based
on a “snap-shot” of the world, and is therefore incompatible
with dynamic tasks and environments. Task planners require
a single model of the world, whereas different models
may be appropriate for different objectives, e.g., an octree
representation is efficient for obstacle avoidance, whereas a
RAPT-style model [36] is better suited to planning compliant
motion tasks. Attempts to develop experimental task planner
systems revealed additional difficulties, such as image understanding,
sensor fusion, error recovery, and the potential for
catastrophic failures [29].
Several intelligent control approaches have been developed
that enabled planning to be performed on line in real time
so that planning became a high-level process within a hierarchical
control system. Examples include mobile robot control
[ 121, factory production control [2], robot assembly [42],
and robotic ping-pong [4]. The hierarchical intelligent robot
control approach proposed by Saridis [38] and Ish-Sh’alom’s
funnel algorithm [25] may also be placed in this intelligent
control category. These controllers are all characterized by a
functional decomposition so that sensing, planning, and control
may be implemented as independent and concurrent modules.
Although these intelligent control methods can be used
for dynamic tasks, they still suffer from many difficulties and
limitations, e.g., sensor fusion and lack of common sense reasoning
and graceful degradation.
’
1042-296X/90/06OO-0357$01 .OO 0 1990 IEEE
358 IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 6, NO. 3, JUNE 1990
A behavioral approach to intelligent robot control was inspired
from observations of simple biological systems that perform
intelligent and complex activities, e.g., reproduction and
hunting, in dynamic and hostile environments. A key hypothesis
was that an insect’s apparently complex behavior results
from the interaction of a small set of simple concurrent perception/
action control processes with a complex environment
[8]. Rather than represent knowledge and reason explicitly as
in the task planner approach, knowledge and reasoning can be
implicit in the parallel decomposition and associated architecture
of a behavior-based system.
Several parallel architectures have been proposed to support
concurrent control processes to perform robot tasks. The
simplest category is decoupled architectures that require a task
decomposition into independent and decoupled processes. The
hybrid position/force controller [35] is an example of a decoupled
parallel architecture, where each degree of freedom of
the robot’s endpoint is controlled by an independent process.
Selection of a compliance frame [32], [39] and its use for
decoupling a task into orthogonal control problems is an excellent
first stage in parallel decomposition of manipulation
tasks but is not sufficient on its own for unstructured dynamic
environments.
A more complex form of decoupled architecture was developed
by Raibert [34] and applied to robot locomotion. He
decomposed the control problem for a single leg into three
concurrent control systems that independently controlled hopping
height, forward velocity, and body attitude. This decomposition
resulted in a simple system design in which each process
was assigned different sensors and actuators. The three
processes were synchronized with the global behavior of the
leg using a finite state machine. The system was easily extended
from two to three dimensions and then to two- and
four-legged machines. Donner [ 141 developed a programming
language for the specification and execution of concurrent processes
and applied it to the control of a multilegged robot using
Raibert’s decomposition.
Brooks [9] defined a subsumption architecture in which
task-achieving behaviors are arranged hierarchically such that
a high-level behavior may inhibit or suppress signals with a
low-level behavior. Each behavior may have its own perception,
modeling, planning, and execution subsystems and must
produce some externally observable behavior. The architecture
is easily extensible, each process is relatively simple,
and different sensors may be assigned to different behaviors,
thereby eliminating the sensor fusion problem. The subsumption
architecture was applied to the control of mobile robots
and provided an elegant and robust solution to navigation and
obstacle-avoidance tasks [8], [ 101. However, the architecture
is inadequate for manipulation tasks since only one behavior
controls the actuators at any one time, whereas compliant
motion tasks require fully concurrent control processes.
Motor schema concepts derived from cognitive science have
been applied to mobile robot navigation and grasping tasks,
resulting in concurrent behavioral controllers [5], [24]. A motor
schema in a robotics context is usually defined as a generic
behavior or control system or more formally as “a generic
specification of a computing agent” [31]. When a schema is
~
’
instantiated, a copy of the generic schema is created, parameterized,
and then activated. A schema-based architecture is
more flexible than Brooks’ rigid hierarchical ordering and facilitates
programming, but so far, there has been little real
world experimentation of the proposed controllers.
This paper investigates parallel decomposition of manipulation
tasks that involve interaction with a dynamic environment.
A methodology for developing concurrent process controllers
for these tasks is outlined in Section 11. The methodology was
applied to a robotic sewing task-the task was decomposed
into concurrent control processes, and a robust controller was
developed. The task, which is described in Section 111, requires
a robot arm to manipulate a fabric panel in conjunction
with a conventional sewing machine to produce an edge
seam. The decomposition of the task is introduced in Section
IV. Development and implementation of the concurrent
processes are described in Sections V, VI, and VII, followed
by discussion and conclusions. The controller was based on a
superiority architecture and exhibited good robustness in the
face of dynamic uncertainty.
11. DECOMPOSITOIFO MN ANIPULATTIOANS KS
Although we perform dynamic manipulation tasks with ease
and skill, simple observation does not provide the insight
needed to program a robot to perform such tasks. True understanding
of the task must be achieved through a combination of
experimentation, theoretical modeling, and analysis. A suitable
methodology for the decomposition of manipulation tasks
into concurrent processes is based on three steps- identify
multiple concurrent objectives, define a compliance frame for
each manipulator in the system, and plan an incremental development
strategy.
The parallel architecture associated with this methodology
is an extension of the decoupled architecture of the hybrid POsitiodforce
controller. More than one compliance frame may
be defined to facilitate task decomposition, and more than one
process may control a specified degree of freedom. Multiple
processes that control the same motion component may be
combined through either subsumption or superposition.
A. Step 1: Objective Decomposition
A complex objective may be decomposed into simpler ones,
leading to a hierarchy of objectives. The number and variety
of objectives will depend on the task complexity and on the
degree of structure and certainty in the environment. Identification
of task-related objects can be difficult, and generally,
one cannot identify all of them from the outset. Incremental
experimentation can help to uncover the inherent parallelism
in everyday tasks. Once a list of objectives has been obtained,
it should be hierarchically ordered to show the relative priority
of each task objective. Objectives may be classified as either
motion limiters or motion generators.
B. Step 2: Compliance Frame
The compliance frame, which was defined formally by
Mason [32], divides the task configuration into positioncontrolled
and force-controlled directions. Both sets of directions
are orthogonal and complementary. The frame may be
GERSHON: PARALLEL PROCESS DECOMPOSITION OF A MANIPULATION TASK 359
fixed relative to an object, to the end effector, or to the world
reference frame, or it may be defined dynamically with its
own trajectory. Kinematic models of manipulation tasks that
relate the compliance frame to the end effector were developed
in [39]. A separate compliance frame may be defined
for each independently controlled subsystem, such as manipulators
and machine tools, to facilitate decomposition of the
task. By relating the objectives to a compliance frame, we
ensure that when motion is blocked along one axis, motion
along the other axes and the behaviors associated with those
directions are unaffected.
C. Step 3: Development Plan
An advantage of the behavioral approach to robot control
is that a controller may be developed incrementally [9]. A
low-level process is designed, implemented, and debugged,
and its performance is evaluated before the next behavior is
considered. An incremental development strategy should be
planned according to the following guidelines:
Assign, where possible, each objective to a specific degree
of freedom of the compliance frames selected in
Step 2. This may not be possible for objectives that are
best controlled in configuration space or joint space since
they cannot be directly decoupled by a Cartesian frame.
Assign a process to each motion generator objective. A
limiter objective may either be integrated into other processes
that control motion along its specific direction, or
it may be implemented as a separate process that blocks
the other processes through subsumption.
Develop the processes in the order determined by the
motion generator objectives such that incremental implementation
is possible.
The task decomposition determined before the development
phase is of a preliminary nature and is intended to provide
guidance later on. During the development phase, more objectives
may be identified, and additional processes may be
defined. For each process, one should initially implement a
simple controller. As inadequacies are uncovered and analyzed,
rather than increase the controller’s complexity, one
should consider additional concurrent processes that would
complement the first and correct its deficiencies. Typically,
an additional process will involve a different sensory input
and will be either superimposed or subsumed on the first.
III. ROBOTISCE WINGA- DIFFICULMTA NIPULATITOANS K
Manipulation of limp fabric is difficult due to its unpredictable
dynamic behavior. Unless it is fully constrained, fabric
may buckle, curl up, distort, or fold over during manipulation,
and its mechanical properties are highly nonlinear.
Conventional automatic sewing systems employ a wide variety
of mechanical devices that constrain the fabric workpiece
to behave as a rigid panel. However, these devices restrict
the equipment to one or two specific operations and often require
extensive manual adjustments each time the fabric type,
size, or shape of cloth panel is changed. Consequently, a more
flexible automation approach using a sensor-based robot is
desirable.
Although several research projects on robotic sewing have
been announced [6], [33], few have published results. A
vision-guided robot was reported [46] that simulated production
of an edge seam. A static image of the fabric panel was
obtained using an overhead camera and was used to plan the
robot trajectory. Despite the use of a very stiff denim fabric
and the absence of a sewing needle or of any fabric feed
mechanism, large seam width errors were obtained when the
robot slid the panel past the simulated needle. The authors attributed
these errors to camera resolution, although the poor
accuracy of the PUMA robot must have contributed significantly.
They concluded that sensory feedback would be necessary
to achieve accurate seams with a real sewing machine
on limp fabric.
In the TC2 project [7] developed at the Draper Laboratories!
a coat sleeve subassembly was produced by a robotic
handling module and an XY automatic sewing module. A
vision system was used as a static sensor to plan robot and
sewing machine trajectories, but real-time sensory feedback
was not used in either module. The fabric was constrained by
a complex system of belts during the sewing operation, and
both robot and sewing module were position controlled.
In the flexible intelligent garment assembly robot (FIGARO)
project described in this paper, a more ambitious
approach was adopted in which the robot was given the central
role of performing both handling and sewing operations
in conjunction with a conventional sewing machine. No hard
automation devices were included that might limit the system’s
flexibility. One of the most common sewing operations
in clothing manufacture is the edge seam-a seam is sewn
along an edge of a fabric panel at a constant distance from a
contoured edge. This task, which was the first to be developed
on the FIGARO system, was a useful and challenging application
on which to investigate robustness and sensor-based
control issues. The general arrangement of the FIGARO system
for the edge seam task is shown in Fig. 1. A PUMA
robot holds the end of a fabric panel against the table using
two spring-loaded fingers while the sewing machine pulls the
fabric forward and produces the seam.
The FIGARO system and its block diagram are shown in
Figs. 2 and 3. The cell controller was an IBM AT operating
under the AMX 86 real-time multitasking executive [l],
and the robot was a PUMA 560 operating under VAL I1 [40].
The IBM AT controlled the motion of the robot using the VAL
I1 ALTER facility. An end effector was constructed with two
spring-loaded, rubber-tipped fingers, which could be raised
or lowered individually by pneumatic actuators. One of the
fingers was instrumented with a force sensor that had a sensitivity
of 1.27 mV/N in one direction only, a cross-sensitivity of
2%, and negligible hysteresis. Fig. 4 shows the instrumented
finger in which four strain gauges were bonded to two parallel
slender beams. Two miniature CCD cameras were mounted
on the sewing machine. The second programmable actuator
in the system was a conventional drop-feed lockstich sewing
machine that was interfaced to the IBM AT. The sewing machine
was mounted on a sewing table that was faced with a
360 IEEE T 'RANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 6. NO. 3. JUNE 1990
Fig. 4. FIGARO robot sewing system.
Fig. 1. Edge seam task
fccd 4
tiacker
SEW
SEWING +
MACHINE
Name Sensors Process Acrunrors
Fig. 2. Concurrent process decomposition of the edge seam task.
IBM AT ~ ccll conmllcr L - 7
machme
camera no camera no
smsoxs ACJL .IJ
<jr(Fig. 3. Block diagram of FIGARO system
smooth and reflective stainless steel sheet. The smooth surface
helped reduce any buckling tendency in the cloth panel, and
its mirror finish provided high contrast for the vision system.
The basic assembly sequence for the edge seam task comprised
the following steps:
1) The robot, using two photocells, finds a cloth panel
placed arbitrarily on the sewing table and places the
two fingers optimally along the top edge of the panel,
pressing it against the table surface.
The robot slides the cloth panel up to the sewing machine
and places it with the needle over the seam start
position. The robot removes its fingers and places them
optimally on the far end of the cloth panel.
The sewing machine is started, and the cloth is fed past
the needle by the sewing machine's feed mechanism.
The robot moves forward with the fabric and rotates
it.
When the end of the seam is detected, the sewing machine
is stopped.
2)
3)
4)
The development of the setting up procedure described in
stages 1 and 2 and of other fabric handling skills are described
in [ 181 and [ 191. The development of the edge seam operation
is described in the following sections.
The problems involved in robotic edge seam production
are considerable. The dynamic behavior of the fabric is governed
by interactions with the robot hand, sewing machine
feed mechanism, and table surface, in addition to its highly
nonlinear mechanical properties. Poor fabric control can cause
unacceptable puckering of the seam. Due to the unpredictability
of the sewing operation, the robot's trajectory cannot be
preplanned but must be entirely determined by sensory feedback.
Both contact and noncontact dynamic sensors are unnecessary,
which could lead to a sensory fusion problem.
IV. DECOMPOSITOIOF TNH E SEWINTGA SK
For the edge seam task, the environment, excluding the
fabric, was static and well structured so that dynamic obstacles
could be assumed absent. The following preliminary list of
objectives was compiled:
a)
b) produce a pucker-free seam
c)
d) avoid obstacles and singularities
e)
At the outset of the project, it was difficult to anticipate how
best to prevent seam puckering since it is a poorly understood
phenomenon [2 11. Pucker cannot even be measured reliably,
but it is known that the fabric's mechanical properties, thread
tension cycle, feed mechanism, cloth tension, and buckling of
the panel all contribute to the problem. The FIGARO system
Produce a seam of constant width
operate and control the sewing machine
avoid torque limits of robot joints.
b
I
could only influence the last two of these factors: therefore.
objective b) was further decomposed as follo\rs: f ) control the
cloth tension and g) prevent buckling of the cloth panel.
Aesthetically, a puckered seam is more objectionable than
a seam that does not follow the edge contour accurately. This
and other considerations led to the follo\ving priority ordering
of the list of objectives (from high to low priority): e), d), c),
g), f), and a).
The robot arm and the sewing machine‘s feed mechanism
constituted the two actuator subsystems. The sewing machine
had only one degree of freedom (along the sewing direction),
and the arm was constrained to three degrees of freedom while
sliding along the table. The pseudo-static task geometry consisted
of a panel constrained to slide on the table and pivoted
about the sewing needle. A compliance frame was defined
with the origin at the sewing needle. the XY plane co-planar
with the table, and the X axis co-linear with the sewing direction.
This choice of frame facilitated decoupling of objectives
concerned with the motion of the panel since the frame was
aligned with the degrees of freedom of the panel and of the
sewing machine’s feed mechanism. Objective a) was assigned
to the rotation of the fabric about the needle. Although f)
is related to the motion in the direction of sewing only, g)
is concerned with both rotation and translation of the panel.
Therefore, objective g) was resolved along the compliance
frame into two objectives: h) Prevent translation-related buckling
and i) prevent rotation-related buckling.
Since the environment is static. objectives d) and e) were
motion limiters and are most easily controlled in configuration
space and joint space. respectively. In the FIGARO system,
conservative empirical approximations were used to derive
motion limits in Cartesian space for avoiding singularities
and torque limits. This permitted simple implementation of the
limiters within each process that controlled the arm’s motion.
The final parallel process implementation of the edge seam
task is outlined in Fig. 5. The SEW process was concerned
with the control of the sewing machine (objective c)) and was
developed first. The FEED process was developed to produce
straight seams by moving the robot hand forward with the
panel. FEED was designed to prevent buckling due to stresses
applied in the direction of sewing (objective h)) but performed
poorly. The TENSION process was associated with objective
f) and was developed to complement the FEED process and
correct its shortcomings. Both processes controlled motion in
the same direction, and their outputs were superimposed. The
CONTOUR process was developed last to enable the robot to
rotate the fabric panel to satisfy objectives a) and i). Limiters
d) and e) were incorporated within the design of the FEED,
TENSION, and CONTOUR processes.
V. THE SEW ANL) FEED PROCESSES
The SEW process started the sewing machine and gradually
accelerated the sewing speed to a desired steady value. A
photocell mounted on the sewing head detected when the end
of the panel approached the needle and provided a termination
predicate for the sew process to decelerate and stop the
selving machine. Although this process produced a puckerfree
seam. there was no control over the panel’s orientation;
ths unconstrained cloth panel rotated during sewing due to a
Fig. 5. Cloth tension sensor.
moment exerted by the feed mechanism on the panel about its
centroid.
To constrain the panel, the robot was used to hold the edge
of the fabric against the table using two spring-loaded fingers,
and a second process, FEED, was developed to move the robot
forward with the cloth feed velocity. This process used the
sewing machine’s shaft encoder signal to compute a cloth feed
velocity and commanded the fabric to move at the same speed.
A proportional relationship between the cloth feed and sewing
machine shaft velocity was assumed, and a proportionality
constant that gave good cloth feed tracking performance was
found experimentally.
The two simple processes developed so far produced
straight seams, but the system’s performance was unreliable.
The proportionality assumption was not accurate, and for different
sewing speeds or different fabrics, a different proportionality
constant had to be determined. Even a small error
between cloth feed and robot velocities could have a disastrous
affect on seam quality. When the robot lagged, the cloth tension
would rise gradually until the seam would pucker. When
the cloth feed lagged, the robot fingers would push the fabric
causing it to fold up, resulting in a faulty seam. The cause of
these problems was the nature of the feed mechanism, which
alternately pulls and releases the fabric. The fabric panel slips
during the release phase, and the amount of slip is dependent
on fabric type, sewing speed, table friction, and cloth tension.
Rather than improve the feed process (through improved
modeling) or replace it with another, the TENSION process
was developed to run concurrently with it.
VI. THE TENSION PROCESS
A. Control Design
A model of the system and its corresponding bond graph
[37] is shown in Fig. 6. The system has two inputs: the sewing
machine velocity e, and the commanded robot velocity xo.
The robot’s position control system is modeled as a mass362
IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 6, NO. 3, JUNE 1990
t- t-r
C : k, R : b. C : k, R : b,
1 T I I I I
Sf + 0 A 1 Y 0 +TF * S f i R : bf
Fig. 6. Model of TENSION open-loop system.
spring-damper system using the principle of mechanical equivalence
[22], which states that for any controlled mechanical
system, a passive mechanical system may be synthesized with
an equivalent dynamic behavior. m, , k,, and b, are the apparent
inertia, stiffness, and damping parameters of the robot’s
position control. The fabric is modeled as a nonlinear damped
spring with stiffness k, and damping b,, and the table friction
bf acting on the fabric is included in the model. The feed
mechanism is assumed to produce a cloth velocity i, proportional
to the sewing velocity by a constant ratio n. Note that
the fabric parameters are functions of time, since the length of
fabric under tension decreases with time, and that the robot’s
parameters are functions of arm configuration.
Since there is no inertia in the robot’s environment and
fabric cannot sustain compressive forces, the robot’s environment
has the characteristics of a mechanical impedance,
i.e., it produces reaction forces to arbitrary input velocities.
Causality considerations dictate that a robot in contact with
an impedance must be controlled as a mechanical admittance
[3], [22]. This implies that a force feedback control is required
in which robot velocity xo is computed as a function of force
error.
Design of a conventional signal process control for the plant
in Fig. 6 would be difficult due to the noisy cloth tension
signal, long time delays, configuration-dependent robot parameters,
nonlinear fabric properties, etc. In this concurrent
pkocess system, the TENSION servo control only had to complement
the FEED process, thus greatly simplifying the design
problem. The FEED process provided gross tracking of the
cloth feed that allowed tracking errors to accumulate, causing
large tension errors. To ensure pucker-free seams, the
TENSION process only had to reduce long-term drift of the
cloth tension by computing a correction to icw,h ich is the
estimated cloth velocity output from the FEED process, as a
function of the measured tension error g ( f e ) . Since i, was
a slowly changing variable compared to f e , the TENSION
servo was designed assuming a constant xc .
I
.
If the outputs of the FEED and TENSION processes were
simply added, the TENSION control would not be robust over
a range of sewing speeds since a specific tension error would
require only a small correction of robot velocity at slow speeds
but a large correction at high sewing speeds. By taking the
TENSION output as a proportional correction to xc rather
than an absolute correction, the system’s performance would
be less sensitive to variations in sewing speeds. Thus, the
control equation for the combined TENSION and FEED processes
was
The input-output description of the plant was obtained experimentally
by measuring the system’s open-loop frequency
response. A long fabric panel was placed in position, and the
SEW and FEED processes were started. After the peak cloth
tension reached a steady value, a sinusoidal forcing function
was superimposed onto the robot’s forward motion, and the
resultant tension variations were recorded. The raw data was
noisy due to random tension variations, and the amplitude and
phase angle of the forced tension variations were extracted
using autocorrelation.
A light tightly woven cotton plain-weave fabric was selected
for developing the servo because of its high pucker sensitivity.
All the experimental results are summarized in the Bode
plot in Fig. 7, which represents an experimentally determined
model of the open-loop system linearized about a specific operating
point.
To reduce long-term tension errors, the tension control
function g was chosen to be a proportional and integral compensator,
i.e., g ( f e ) = K, f e + K i C f e , which resulted in
the following equivalent compensator transfer function (for
constant ic):
GERSHON: PARALLEL PROCESS DECOMPOSITION OF A MANIPULATION TASK
3
Measured Tension
Robot Hand
-k UPDATE RATE HALVED
363
8 0 , , , , I , I 1
X 2 PLY PANEL, GAINS HALVED
-
- 0 ALONG ?HE BIAS
-
-
0 " " ~ " " ' ~ " ' ~ " " " " "
Pressure
Finger Actual Cloth Tension - Table Friction
Fig. 7. Effect of table friction on apparent cloth tension.
A derivative gain was not appropriate since it would amplify
tension measurement noise and would only influence
short-term errors. Ballpark values for the compensator's gains
K, and K; were calculated using the graphical Bode design
method and conventional stability criteria, and they were finetuned
using trial-and-error experimentation.
B. Performance of Straight-Line Sewing
Robot force control implementations exhibit instability
when in contact with a stiff environment. Time delay, noncolocation
of the force sensor, unmodeled dynamics, friction,
and backlash in the joints are some of the problems contributing
to this instability [16], [47]. In our sewing application, the
environment was very soft, and these problems were less significant.
However, the poor tension signal, the table friction,
and the nonlinear tensile properties of the fabric presented additional
difficulties. Woven fabrics exhibit significant hysteresis
under tensile loading and have a strain-dependent modulus
of elasticity. These characteristics, in combination with the intermittent
cloth feed motion, resulted in limit cycling of the
cloth tension, even for slow speeds and low gains. However,
as long as the tension fluctuations were small, the seam was
pucker-free.
The velocity, acceleration, and jerk of the end effector were
limited to prevent jerk-related buckling of the fabric. The
sewing machine's acceleration was limited within the SEW
process to improve the TENSION control. Within the FEED
and TENSION processes, the robot's velocity was limited to
the forward direction only (i.e., towards the sewing machine)
to minimize the effect of the table friction on the tension measurements.
As shown in Fig. 8, the friction force changes
direction when the hand motion changes direction, causing a
large change in the apparent cloth tension.
Seam quality cannot be measured or quantified, and therefore,
experimental results cannot be presented in terms of
seam quantity. Generally, standard deviation of tension error
gave a satisfactory indication of seam quality since an increase
in standard deviation accompanied a reduction in stability margin,
which lead to degradation of seam quality. Fig. 9 shows
the performance of SEW, FEED, and TENSION processes
producing straight-line seams.
For a single-ply panel of the test fabric, good quality seams
were obtained when the standard deviation of tension error
was 15 gf or less. Two-ply panels and heavier fabrics had
a reduced buckling tendency, and they produced high-quality
seams for a standard deviation of up to 30 gf. Fig. 9 shows that
the servo's performance can be characterized by a maximum
feed velocity V,, at which there occurs a sharp degradation
1 5 10 50 100
FREOUENCY (RADSIS)
Fig. 8. Bode plot of TENSION process open-loop frequency response.
SEWING SPEED (rpm)
Fig. 9. Straight-line sewing performance.
3f servo performance. For velocities below V,,,, the system
was robust to changes in fabric length, table friction, and humidity.
Different categories of fabric (e.g., lightweight and
:ightly woven or lightweight and loosely woven) required difrerent
values for K; and K,. Increasing the number of plies
.ncreased the panel's stiffness and decreased the correction
%stance necessary to obtain a specific tension increase. To
:ompensate, the gain values were divided by the number of
>lies, and this resulted in a similar performance as for the
3 64 IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 6, NO. 3, JUNE 1990
single-ply panel. When the robot motion commands were only
updated every two ALTER cycles (i.e., every 56 ms), V,,,
was reduced from 2400 to 2000 r/min (Fig. 9).
In all the tests described so far, woven fabric was sewn
approximately along the warp or weft directions. When the
fabric was sewn “along the bias” (i.e., at 45” to the warp),
good tension control was obtained at all speeds due to the
much lower stiffness of the fabric in the bias direction, which
effectively reduced the system’s apparent gain and improved
stability. When the fabric is pulled along the bias direction
instead of extending the fibers, the fabric structure is distorted,
and this results in a complex three-dimensional buckling of
the panel. A similar phenomenon occurs with knitted fabrics
along any direction. Although this form of buckling was not
along the sewing direction and did not cause pucker, it did
interfere with the CONTOUR process.
VII. THE CONTOUR PROCESS
To extend the system’s capability from sewing straight
servo process called CONTOUR was developed. With a conventional
drop-feed sewing machine, the panel is constrained
so that only rotation about the sewing needle can modify the
seam width during the sewing operation. The CONTOUR process
comprised two modules: A contour tracking module measured
the edge contour in the region of the sewing needle and
computed an angular velocity w at which the fabric should be
rotated, and a motion-planning module computed the robot
motion required to perform the fabric rotation.
A. Contour Tracking
Analysis of the control problem was only feasible for a
single-pixel bang-bang controller, which was proved to be always
unstable [18]. Consequently, a simulation of the CONTOUR
process was developed that included time delay, robot
reach, velocity limitations, and the vision system’s resolution
and field of view but assumed a stiff cloth panel. Simulation
studies provided insight into the control problem and showed
that stable control could be achieved with a control law based
on feedback of E, which is the seam width error, and p, which
is the angle between the contour tangent and the direction of
sewing. The control law was
I seams to seams that follow the edge contour, an additional
“
The simulation model was used to design the vision system.
More details on the simulation model and on the development
and implementation of the CONTOUR process was given in
~ 9 1 .
B. Motion Planner
To rotate the cloth panel about the needle, a robot motion
that comprised two motion elements was required: The
instrumented finger was rotated about the needle, and the auxiliary
finger was rotated about the instrumented finger (Fig.
10). By defining the robot’s TOOL coordinate frame at the
instrumented finger and by referring the motion command pa-
Sewing Needle
Yneedle
end-effector
zneedlc
Fig. 10. Rotation of cloth about needle.
element was obtained by setting TO, which is the robot motion
command for rotation about the z axis, equal to w. The values
of the other motion command parameters xo and yo required
to rotate the main finger about the needle were derived as
follows:
X: =Rcos(a+wT)
= R (cos a - WT sin a) for small wT
= Xr -@Tyre
Similarly
Hence
xo = -wyr (4)
yo =WXr (5)
where (xr, yr) and (xi, yi) are the positions of the TOOL’S
center point relative to the compliance frame before and after
rotation, a is tan-’ (yr/Xr), Tis the servo time delay, and R
is the needle to TOOL distance. This derivation ignores the
motion of the robot and cloth due to the feed mechanism, and
the final motion command in the direction of the sewing was
the sum of the ko output of the CONTOUR process and the
xo output of the FEED and TENSION processes.
The robot could not always produce the motion demanded
by the contour tracking module due to work envelope limitations
(including singularity regions) and due to the low buckling
strength of fabric. The velocity, acceleration, and jerk of
the end effector were limited since it was found that smooth
and damped motions of the arm reduced the fabric’s buckling
tendency. Although all three motion parameters xo , Lo, To
required limiting, the yo component would usually hit a limit
first due to the arc-like nature of the robot motion, and there-
*
rameters to the WORLD coordinate frame, the second motion fore, the limitations were applied to yo first. The limitations
GERSHON: PARALLEL PROCESS DECOMPOSITION OF A MANIPULATION TASK
were applied in the following order: velocity, acceleration,
jerk, work envelope; therefore, each limiter would reduce the
likelihood of the next one limiting. When the robot was approaching
an envelope limit, a velocity profile limiter was
applied to yo such that the robot would smoothly decelerate
up to the boundary rather than stop with a jerk. If, after application
of the limitations, yo had been changed, 1, and To
were then recalculated using (5) and (6) to maintain the correct
relationship between the components, thereby keeping the
fabric flat and undistorted. This limitation procedure was then
repeated for xo and To.
During sewing, the robot's hand was always flat on the table
with the two fingers facing the sewing needle, and these
constraints enabled further simplification such that both obstacles
and singularity regions could be plotted on the twodimensional
chart shown in Fig. 11. Boundary A was set to
prevent collision of the end effector with the robot's base as
well as to avoid a wrist singularity region. The B boundary
prevented the robot from moving past the left edge of the table.
Boundary C represents the elbow singularity region (arm
fully outstretched), and boundary D prevents collision with
the sewing machine. Boundaries A and C were referred to
the position of the robot's tool flange, and D was referred to
the position of either of the two fingers.
C. Performance of Edge Seam Sewing
Extensive performance tests were performed on the test
fabric to determine the sensitivity of the system's stability to
speed, gain values, and number of plies. Exploratory tests
were performed to investigate sensitivity to fabric type. The
edge contour used in all the tests had a moderate curvature
that was representative of contours found on trouser, jacket,
and skirt panels.
A standard deviation of E of 0.6 mm was chosen as the
performance criterion on which Figs. 12 and 13 are based.
The standard deviation of was an appropriate measure of
performance since a wavy seam is unacceptable aesthetically,
whereas a small constant offset error is acceptable. The performance
envelopes indicate regions within which the performance
criterion was satisfied. Fig. 12 shows the effect of cloth
speed on task performance for two-ply panels, and Fig. 13
shows the effect of the number of plies on task performance.
Good quality edge seams were obtained with the two-ply test
panel for speeds up to 100 mm/s, which is comparable to
the performance of skilled operators. Stability margin was
improved by reducing the fabric's buckling tendency through
reducing the sewing speed, increasing the number of plies, or
by using a heavier fabric.
The robot's accuracy was only 7 mm rms for xo and yo
and f 0.2" for yo, and this caused buckling of the fabric during
open-loop rotation of the panel about a stationary needle.
However, during sewing, the fabric did not buckle for the
same rotation since the TENSION and CONTOUR feedback
servos tended to accommodate the robot's inaccuracy.
The maximum contour that could be tracked was primarily
a function of the robot's reach and the initial position of the
fingers. When the fingers were closer to the needle, it was easier
to rotate the fabric and track contour changes. A CLOSE
365
SEWING TABLE
Fig. 11. Work envelope for edge seam task
i 1 i
i 1
i i 1
0.00
0 0.01 0.02 0.03 0.04 0.05
K1
Fig. 12. Effect of sewing speed on edge seam sewing performance.
0.10
0 . 0 5 1
K2 i
t x ONE PLY
0.00 ' ' ' ' ~ 0 0.01 0.02 0.03
K1
Fig. 13. Effect of number of plies on edge seam sewing performance.
366 IEEE TRANSACTIONS ON ROBOTICS AND AUTOMATION, VOL. 6, NO. 3, JUNE 1990
sewing technique, which permitted FIGARO to sew around
sharp contours, is described in [19]. Optimum positioning of
the fingers at the start of the seam task is investigated in [IS].
VIII. DISCUSSION
When the extent and variety of environmental uncertainty
involved in a robot task is greater, the number of control
objectives that must be considered is larger as is the number of
sensors that are required to achieve those objectives. Several
types of uncertainty were involved in the sewing task, such as
the unknown edge contour, the unpredictable behavior of the
fabric panel under a complex combination of tensile and shear
stresses, the poor static and dynamic accuracy of the PUMA
robot, and the unpredictable performance of the feed mechanism.
The decomposition of a complex control problem into
a set of simple decoupled ones is a well-established technique
in the repertoire of the control engineer. This paper indicates
how this technique can be applied to robot manipulation tasks
and extends it to produce a decomposition comprising interacting
servo loops within a parallel process architecture.
SEW, FEED, and CONTOUR were decoupled processes
with respect to the compliance frame, whereas the TENSION
process was superimposed onto the FEED process. Although
the FIGARO implementation did not fully exploit the inherent
concurrency in the parallel decomposition of the task, the
SEW, FEED, TENSION, and CONTOUR processes could
all be implemented as independent processes on separate processors
running at different update rates, and they require
no synchronization. Even though the TENSION process, as
shown in Fig. 5, is dependent on the output of FEED, the
communication between the two processes does not have to
be synchronized since x, changes gradually relative to the
varation in tension measurements, and therefore, the FEED
process can use quite old values of xc without detriment.
The conceptual parallel architecture depicted in Fig. 5 has
not been generalized or formally defined. Several problems
such as torque limiting and obstacle avoidance were bypassed
through conservative simplifications. This work represents a
step towards defining a general architecture for intelligent
robot controllers that might also include aspects of Brooks’
subsumption scheme and Khatib’s approach to obstacle avoidance
and torque limiting [26]. Advancement of a control-based
approach to intelligent robot systems hinges on the development
of parallel process architectures.
The guidelines outlined for the decomposition of manipulation
tasks do not constitute a formal methodology but should
provide some assistance to the developer of intelligent manipulation
systems. The emphasis throughout Sections V and
VI has been on the incremental development that combined
experimentation with modeling. The final decomposition is
mainly a result of the development process, whereas the initial
task analysis, which is described in Section IV, provided
the development plan only.
The FIGARO system was based on simple linear control
systems, but more sophisticated controllers within the same
parallel decomposition would offer significant performance
benefits. For example, the present system’s sensitivity to fabric
type and sewing speed could be reduced by modification
of gain parameters either through on-line adaptation or by using
table look up. However, some of the system’s unsolved
problems would be best tackled by further behavioral decomposition.
For example, improving the TENSION control is
unlikely to extend the system’s capabilities to knitted fabrics
or to along-the-bias sewing. This is because tension measurements
do not reliably indicate fabric distortion due to hysteresis
and fatigue effects in the fabric. An additional concurrent
process based on a different sensor, such as a laser range
sensor to detect folds in the fabric, would be a more viable
approach. System robustness might be further improved by a
watchdog process that would sense the onset of servo instability
and reduce the sewing speed accordingly.
The system demonstrated robustness to uncertainty by producing
good-quality edge seams under a wide variety of
sewing speeds, panel contours, number of plies, and fabric
type. The system also accommodated uncertainty in the position
of the robot endpoint due to the poor accuracy of the
robot. Much of this robustness derives from the low gains
that were applied to the servo feedback loops. The parallel
decomposition enabled low gains to be used without compromising
the performance of the high-level task since the lowlevel
objectives had low performance requirements, e.g., only
moderate tension control was necessary to ensure high-quality
seams. Note that there were no explicit models or expert production
rules to help the system decide how to constrain the
fabric, but that knowledge was implicit in the decomposition
of the task and in the design of the processes.
IX. CONCLUSION
A parallel process decomposition of a dynamic manipulation
task has been described. The decomposition endowed
the robot system with robustness, simplified the design of the
control systems, and eliminated sensor fusion problems. The
decomposition was found by an incremental development strategy
that combined modeling, analysis, simulation, and experimentation.
The processes were designed so that outputs could
be superimposed to produce the desired robot trajectory.
The viability and potential of parallel decomposition of manipulation
tasks has been demonstrated, and other tasks, such
as two-handed assembly, could be tackled next. The conventional
sequential interpretation of manipulation tasks should
be replaced with one based on concurrent processes, and this
requires identification of their inherent parallelism.
[I1