Hello,

We occassionally implement our own XYZ "robots" for specialized high speed and accurate applications. I am interested to see if there is a motion control vendor that can automatically restore to the position that the motor was in when the safeguard was opened, and then continue the motion trajectory it was running from that point. This is very similar to how Epson Robots (and I presume other robots) function "out of the box".

We are finding that simply resetting the errors and re-running the motion program can often times be all you need to do, but there are times where you need to make sure you are clear of tooling, etc, which complicates your motion code alot. For very complex move profiles it can be very tricky to perform fool proof recovery and still make the code understandable 10 years down the road.

I know that many motion controllers can handle this if you are willing to write alot of recovery code, but I'm looking for the most integrated and seamless way of doing it. An ideal method would be if the servos could maintain position with a "safe" low torque setting until the safeguard was closed again. This may have to be hardware controlled in the drive for safety reasons, so maybe this one is more for an integrated system. I sense that the safety gods would throw down lightning over this approach, but it potentially could be made safe.

Thank you,
KEJR

 

I don't remember the details, but I recall something about the Turbo PMAC lookahead buffers and potentially being able to rewind a program or run it in reverse. You might call Delta Tau and see if they can help.

Bill Sturm
Abbeytronics LLC

 

I'm not sure of your exact application, but I assume the problem is operators opening the guard doors when your equipment is not prepared to deal with that. A straight forward way of dealing with this type of problem is to simply lock the guard door with a solenoid lock. If you only allow the door(s) to be unlocked when you are ready to deal with it, you don't have so many recovery points.

I have used this solution in a number of applications where a process cannot be interrupted in mid-cycle without compromising product quality. You can still allow the door to be opened at non-critical points in the cycle, but over-riding the lock (in a safe manner) at critical points would be a non-recoverable error requiring restarting the cycle from the beginning.

 

I had talked to our Delta Tau distributor briefly but no ideas came to mind. Getting ahold of them directly wouldn't be a bad idea. I wasn't aware of the running a program in reverse feature, this sounds like its getting close to what we want.

Thanks,
KEJR

 

Hello,

Thanks for the reply. Solenoid/air cylinder locks are attractive for the case where the machine is opened up while running a move. I have thought of this and still may implement it. In the machines we build it is often necessary for the operator to open a door to clear some kind of jam while the machine is in a run cycle. The servos are not necessarily running at the time, usually the motions will be complete by the time the operator is alerted of the particular jam. At this point they would open the door, clear the jam, and then the cycle would resume without any real intervention from the program other than the typical operator acknowledge from the HMI. In this case the only issue is that the operators in the process of clearing a jam could have moved the servos from their last commanded position. I could, in fact, record the position of the stopped motors upon safeguard open, and return them upon safeguard close. This also gets tricky because there is often interesting ways that a three axis system can crash into things if the order of restoration is not done correctly. In fact the order changes depending on where you are in 3D space because of [often] complex obstructions. Of course there are tricks like torque limiting, slow accel/decel, slow speed, tightening up fatal following error limits, etc, to keep the axes from crashing really hard, etc. Although these approaches do not act like a Robot "Restore and Continue", it does accomplished most of the goals in a different way.

Thanks for the ideas, I will have to come up with a decent solution for the long term with some of,if not all, of these ideas. As Bill suggested I'll look into Delta Tau solution for future projects to see whats possible from those guys.

KEJR.

 

Using our Turbo controllers this is a fairly easy problem. The only wrinkle is that you have to change the way an error is handled (i.e. Fatal Following Error or Amp Fault) so that the motion program in progress is not killed by the safety interlock. You would write some code to handle these faults, and when the interlock opens (which would need to be mirrored as an input to the PMAC) you would pause the motion program, thereby halting all motion, and kill the power to the amps so that no controlled motion can occur (which I assume your safety relay is doing anyway). The operator could potentially manually the robot if necessary. Upon closing the interlock, the start command from the operator would cause the motors to jog to the last commanded position, then you would resume the program. I've done appications like this several times. If you have an exclusion zone, you could handle that a couple of ways. Either have the 'jog' move back to position run through code which handles your exclusion zone, or you could get fancy (which is how I usually handle this) and have two coordinate systems set up, both with your full kinematics including exclusion zones. CS1 contains only phantom motors, and CS2 has your real motors. During ordinary operation, your motion profile is run through CS1, and the motors in CS2 are slaved to the commanded positions of the phantom motors. When you have to halt the program, you can run another motion program on CS2 at restart to take the motors back to position, then restart CS1. If you do it this way you can even avoid the custom fault handling if you want to.

Feel free to contact me offline for more detail.

Davis Gentry
Senior Applications Engineer
Delta Tau Data Systems
d gentry (at) delta tau (dot) com
(remove all spaces and use appropriate symbols)

 

The conflict that comes to mind is:
Robot does something socially unacceptable. Robot wrangler opens door and steps in. Closes door, and robot strains wrangler through fence.

That's why most are set up so you need to home and restart, from outside the fence. Or use a teach pendant, so it's your fault. Robots are very predictable and repeatable. So, if it does something different and faults, there is no guarantee that it was not the first move of a rampage. The last robots I've worked with would kill you without even slowing down. I'm pretty sure you need zero energy (except brakes) with a safeguard open and a considered action to restart. It's not that hard to write to recover to a known safe point from all stored positions and restart. I have seen a robot take off and mash tooling on a restart with nothing in the code like that path. There are things that can be done, but a safeguard needs to be a safeguard. You have to kill the bot. I know management doesn't like downtime, but this is life safety, at least with the bigger robots.

Regards
cww

 

No, I'm not talking about auto-restart. The restart is initiated by the operator, hopefully once whoever has walked inside the fence has exited. You have no bus power to the amps, so straining the wrangler through the fence is rather unlikely. If you use my second method, you can kill the motors (CS2) at the pmac since your motion trajectory is running on the phantom motors.

 

Yes, I see that from later comment. You'll have to excuse me, you wouldn't believe some of the things management had proposed. I hope they didn't get their way with me gone.

Regards
cww

 

I think Ken is asking the robot to move back to the last commanded point using the inverse of the operators manual positioning sequence. I have an idea to monitor the position of the robot hand and store each change in position in a buffer.  Then slowly play back the buffer in reverse order before restarting the program.  Is that correct?  If so, I think a PMAC could perform these functions easily.

Bill Sturm
Abbeytronics LLC

 

If one were to write it like a state machine, recovery could be fairly simple and this could be done even in Karel. To recover set the recovery flag and transition to the last good state. The state, of course would have the stored position and the conditions to transition to the next state. The move would be dependent on the state and a safe path stored from that position or along the path that got it there. That is, the inverse path would be called if the recovery flag were active. Waypoints could be used if the path were complex. But that would simply be more states.

Regards
cww

 

This is all really application dependent. I can remember going through all this (although with different hardware) and making great efforts to be able to restart a process from any point.

Then when we started doing process FMEAs for real (instead of just going through the motions) we discovered it was a waste of time in our applications. The PFMEAs were all designed with the idea that each process was completed without interruption. Once you started allowing for people to do whatever they wanted in the middle of a cycle, the PFEMA scores started failing badly. By permitting arbitrary machine restarts, you are essentially allowing unauthorized rework operations.

The conclusion in the end was that restart/retry had to be examined carefully for each process. In many cases it simply couldn't be allowed. If a process failed in cycle, the part had to go through the approved rework procedures. That was where the solenoid locks (for non-safety use) came into the picture. If you opened the door in-cycle, the part was scrap (or rework). Machine downtime issues were addressed by eliminating the causes of the downtime, rather than the symptoms.

I don't know what your machines do, so I can't comment on how this applies to you. However, you have to look at the big picture before you decide whether this is even a good idea.

 

This is an interesting idea. You could probably just drop flags or increment a counter based on certain positions being reached. I think to do this would require a motion controller with high level control because to do such a thing without function calls could be very repetitive. Currently we are using the Mitsubishi motion controller for their Q rack system, but for applications like this I think we would be better off going back to Delta Tau. The traditional PMACs, and especially the UMAC system are hard to setup (depending on the complexity and configuration of your system) but they tend to be easy and powerful to program once you have your motors up and running. I think the Mitsubishi is somewhat opposite, it is easy to get your motors running (they are somewhat plug and play, autotuning, etc) but the programming is not very powerful (IT is SFC based, not text based).

Thanks,
KEJR

 

Hello,

I'm not proposing anything toward automatic recovery (that is without an operator present), or in the sense of bypassing hardware safety. We use double contact switches going through safety relays to both kill motor mains and issue an ESTOP signal to the amplifier and the PLC (for monitoring).

KEJR

 

Hello,

Thank you, this sounds like exactly what we need. I will be in contact regarding the next machine we do. I also have questions about the Power PMAC. Not sure if it will be released officially in time for the next machine, but we could look at using a Turbo UMAC.

Thank you all, I think we have a winner. If this works I think it will be elegant, easy to use, and very maintainable.

KEJR

 

Setting up a PMAC has gotten much easier in the recent years with the TurboSetup program.  It walks you through the whole process, no I vars are necessary.  They do display them for your review, but you don't need to know them.  It is handy for maintenance too, just run through the setup program and see what works and what doesn't.

Bill Sturm

 

In reply to Davis Gentry: When you said "The operator could potentially manually the robot if necessary", are you talking about physically pushing the axes, or are you talking about jogging it under power?

Is there a straight forward way to allow the operator to jog the machine to any arbitrary "clear" position and then resuming from there? Or was that in fact what you were talking about (in which case I apologize for not paying attention closely enough)?

By the way, my previous posts on this subject weren't intended to throw cold water on the whole idea. The point was that in a lot of applications elaborate recovery procedures are pointless. However, there are lots of other applications where it would be nice to have better recovery options.

 

Michael -

Sorry, I just read this one.

I was referring to manually moving it if required. But you could also, if needed, after restart by the operator jog the robot to some position, including back to the last programmed position of CS1. This "jog" move would be handled in CS2. Since CS1 has never aborted its' program you can resume the program either from the last programmed position, or from any arbitary point, and it will continue on the programmed path from the point where the CS2 motors finish up prior to the program resume. You can also retrace previous points on the trajectory, then resume the program after the retrace.

Davis Gentry
Delta Tau Data Systems