P
Hello All, The following is a copy of an email describing the development of NIST's EMC (Enhanced Machine Controller). It is a long message, but may be interesting to some people on this list. Some of the issues the author discusses at the end of his email are, I think, pertinent to the lPLC. Regards, Phil Covington vHMI ---------------------------------------------------------------------------- ---------------- From: Matt Shaver <[email protected]> Date: Mon May 17, 1999 3:36pm Subject: Re: NIST EMC Software & Intro Hi, I've been using the EMC software for a few years now. I have a little machine shop that has been a part time venture since 1987 and my full time occupation since the summer of 1995. In the spring of 1996, shortly after I bought out my business partner and was in considerable debt, the control on my Bridgeport V2E3 went bad. I didn't want to spend $20K on more proprietary technology that I'd have to scrap at some point down the road when vendor support dried up. I looked at a number of replacement servo based controls such as Proto-Trak, Centroid, Anilam, Bauch & Lomb's MillPower, Milltronics, Lighthouse Software, and probably some more I have forgotten. I even drove up to Centroid's factory in State College, PA to see their wares first hand. I have to say that if I were going to buy an "out of the box" control it would be a Centroid as I believe they offer the best value. Still, even using the old motors, I was looking at $12K for the Centroid and it was still proprietary (although I have to admit they were very nice folks). Anyway, in June of 1996, I was trying to get together a proposal to develop a 3d scanner for a company that made orthotics (corrective shoe inserts). The purpose was to digitize the shape of the bottom of the patient's feet, rather than using impressions in plaster casts. I was researching it on the Internet and found that NIST had done a project on this very subject. Well, to make a long story short the scanner project never panned out for me (they went with another consultant who was going to do it by gray scale interpretation of the Z axis from 2D black and white scans). However, I stumbled across the EMC project while browsing the NIST web site. I called NIST and inquired about getting a copy of the software. I think they were a little skeptical of me at first since their other collaborators on the EMC project were Boeing and General Motors. I persevered and finally got in touch with Fred Proctor and Will Shackleford who did whatever was required to get me signed up as a bonafide researcher. I gutted the Bridgeport's electronics cabinets leaving only the power switch, a few transformers, the lube pump, pneumatic controls and the e-stop circuitry. I installed new servo amps and an Opto22 rack to drive the spindle relays, coolant pumps, etc. In addition, I added a shelf inside one of the cabinets... (At this point in my typing I thought "They're never going to really understand without pictures". So I stopped and made this web page): http://users.erols.com/mshaver/v2e3.htm As I was saying, I made the changes to my mill that you see on the above web page. Meanwhile Fred and Will at NIST were changing the software around to suit my machine. The previous implementation of the EMC at General Motors (a K & T mill with tool changer at the Pontiac Powertrain Division) used two PCs networked together with Ethernet. One PC ran a real time OS (LynxOS) which communicated with the motion control card and digital I/O hardware while the other PC ran a Visual Basic GUI under Windows 3.11. I had expressed my desire for a system that would run on a single PC under Windows 95. Fred said he could do it using Windows NT so that is what he did. Another change was to move the digital I/O from a dedicated I/O card to bits on a parallel printer port. Both these changes were done with cost reduction in mind (my mind anyway!). In August of 1996, I had finished the hardware changes and Fred and Will had finished their software work. We assembled all the pieces, and after a few days of fiddling and fussing, it worked! This is of course a huge understatement, or non-statement, of what was involved but it's the best I can do since lots happened, none of which I remember accurately. During the period from September 1996 through November 1996, we did lots of testing and software revisions, which I downloaded from NIST's ftp server and installed, on the mill to fix this and that minor problem. On November 26, 1996, Fred and Will brought some of the folks from NIST up to my house to have a look at our handiwork. I ran a demo program for everyone that engraved the NIST logo on a piece of 1/16" aluminum sheet. A picture taken that day can be seen at: http://www.isd.cme.nist.gov/projects/emc/emcsoft.html The guy in the plaid shirt staring intently at the vise is me. I think you can just make out the Visual Basic GUI on the monitor. Around the beginning of 1997, Fred and I had a number of discussions regarding where to go from there. One of the problems I had with the setup we had at this point was that it used a PMAC motion control card from Delta-Tau. This is a great board, literally a cnc on a card. It has a dedicated DSP per axis and high-speed dual ported RAM for communication with the host computer. It is totally configurable, both with jumpers and in software. It comes with a comprehensive manual as thick as a phone book and excellent technical support is available. It costs $4000. In the time between the failure of the original control and finishing the installation of the PMAC/NT based control, I had changed by business methods around some. I had to keep on producing parts so I subcontracted all my cnc milling jobs out to a friend of mine. This was actually the best thing I ever did business wise, but that is a subject for another forum. The result was that I could use my mill as a guinea pig for EMC experiments without any pressure to produce work with it. Although once it was running with the PMAC under NT I did do several paying jobs with it and it worked better than it ever did with the original control. At this point, my own thinking had progressed beyond fixing the Bridgeport mill to building more machines. With new imported cnc knee mills in the MSC catalog selling for $20K or so the only way for domestic builders to compete is to get the cost of a complete retrofitted machine down to $15K or less. Any higher than this and potential clients are going to say "I'll just buy new and get a warranty". Let's look at the costs: $6000 Good, tight, worthwhile Bridgeport $1000 Ballscrews (Ground), Paint, Miscellaneous Mechanical Parts $500 X & Y Brackets $1500 Z Bracket & Quill Drive $3000 3 new size 42 servo motors with encoders $2000 Servo Amps (The set shown on the web page above cost $1900) --------- $14000 TOTAL This doesn't include a PC, motion control card, software, or any of the "glue" like the Opto22 rack, relays, enclosures, etc... In addition, if you sell your machine through a salesman (like I have twice) you're going to have to pay a commission, in my case 10%. You might say, "Those costs look a little high to me, I could get the machine cheaper, use rolled screws, surplus motors, make the brackets, yadda, yadda, yadda." but if the motion control card costs $4K you're finished. To be completely fair to Delta-Tau, their card is priced right for what you get and if you were retrofitting a large machine tool that cost six figures the PMAC price would be insignificant in the grand scheme of things. Other potential applications for the PMAC are projects that require fast time-to-market in which case you can purchase a complete, tested, reliable solution that works right away. Also, the PMAC is capable of controlling motion at much higher velocities and to greater resolution than is required in machine tools. Applications such as controlling linear motion stages that are part of the photolithography processes used in the manufacture of ICs for example. In cases like those the host computer may not have enough computing power that it can devote to motion control while still running other tasks like a user interface, file system, etc. I believe that some versions of the PMAC can run stand alone with only a power supply and a terminal. In summary the PMAC is like the Cadillac of motion control cards, but if you're like me you only have enough money to buy a Chevrolet (actually, if you're like me, you don't even have that much money). Really, the best way to go is to buy old nc or cnc machines with dead controls and retrofit those because you can often reuse the motors, amps, etc. In addition, non-running cnc machines are generally regarded as nearly worthless. They typically sell for less than an equivalent sized manual machine. Still, you need to be able to sell cheap to compete against the new import machines and yet make a profit. Just as an aside, you can buy a new import linear way 40 taper vertical machining center for around $30K! If you bargain hard, you may get some tooling (cheap) thrown in and a few months of interest only financing. I think I got off topic here somehow... Like I was saying, after the original NT/PMAC based version of the EMC was stable I expressed a desire for the cheapest possible implementation. I had started reading rec.crafts.metalworking around this time and I thought advanced hobbyists would be interested in building servo-powered systems if there was software available to control them. Anyway, around March of 1997 Fred said he was interested in porting the EMC to Linux and using a simpler (cheaper) interface card. His reasons were that the PMAC card did a lot of the motion control work internally. If the EMC was going to fulfill its original purpose as a platform for developing new, advanced machine tool control methods then all the motion control algorithms should be implemented in software to allow experimenters to tinker with them easily. This meant that the only hardware functions that would be required are DACs to output the control voltage to the servo amps and quadrature encoder readers to feedback position information to the control. Digital I/O was already implemented on a printer port, although the home and limit switches as well as the servo amp enable signals are tied into the motion control card. That would suit applications where the I/O could be mapped into twelve output bits and five input bits per port with a max of three ports on a PC. Fred had already identified the components used in the current implementation; Linux, Real Time Extensions for Linux from New Mexico Tech, and an interface board from Servo To Go. This board is available in several different versions but the full blown 8 axis version costs $888 (< 25% of the cost of a PMAC). More information on RTLinux can be found at: http://www.rtlinux.org Basically RTLinux adds hard real-time deterministic control capabilities to the standard Linux kernal with time resolution down to 100uS. Standard Windows NT is capable of about 10mS resolution, but I/O requests from your control program could be deferred by the OS for several seconds. Additional software products from Venturcom and Radisys that work with NT can reduce the resolution and fix the real-time task priority problem, but are not free (neither is Windows NT or VisualStudio for that matter). The hardware changes to the mill required to use the Servo To Go card instead of the PMAC were minimal. I continued to use the PMAC throughout the summer of 1997 during which time I made parts for an architectural model and a few other things. By October of 1997, the software had been ported to Linux, which was a major undertaking for Fred and Will. It involved not only the port itself but also writing the motion control functions that had been previously handled by the PMAC. There was also the device driver or "wrapper" for the Servo To Go card and numerous other smaller issues to resolve. On October 30th we started installing the Linux based system and there followed a several month long period of testing and software (and a couple of hardware!) revisions. One interesting feature of the Linux implementation was the user interface that was written in Java. Theoretically you could put the GUI applet on a web page and control the machine from anywhere. Unfortunately, in practice, the Java GUI turned out to be a problem because after running the EMC for a period of time (say 10 minutes to an hour) X Windows would freeze and the PC would have to be rebooted. In November 1997 I exchanged e-mail with Jon Elson regarding NFPA and EIA standards applicable to machine tools and I told Jon about the EMC project. He started using the EMC software and has done a lot of troubleshooting and testing. Since he started at the beginning of the Linux code conversion, he has seen the majority of the Linux related development. He also probably has more "time behind the wheel" of anyone operating an EMC controlled machine. At the end of April 1998, I had an offer to buy the Bridgeport mill from a company in York, PA. Their requirements were simple in that they wanted to automate the drilling of a long series of holes in small copper tubes. I wasn't out soliciting offers, but I had shown the machine to a machinery salesman friend of mine. He called a few days later and asked, "You want to sell that mill of yours?" I have to say that at this point the technology was not "ready for prime time". I was however, in nearly desperate need of money and with an offer on the table, I decided to sell the machine with the confidence that I could resolve any outstanding issues that came up. And come up they did, rather like dandelions in the spring. It took about a month and several trips up to York by Fred and I to resolve the outstanding issues. The java interface was replaced with a console program called keystick that Fred wrote using the ncurses terminal control function library. Considerable time was also spent in tracking down and finally defeating the mysterious and intermittent "not a number" problem which caused axis runaways to occur. We still go up there from time to time to install the latest version of code, and to get user feedback. In retrospect, I probably should have waited until the software was more stable before I sold the machine. However, the pressure to fix the problems did speed the development process! Before I sold the mill, I wasn't using it enough to shake out all the bugs. The truth is, software projects need green testers (translation: guinea pigs) because people close to the development tend to overlook problems because they are too close to the process. In this case, the customer got the capabilities they needed for about half of the monetary cost of their nearest alternative solution. They also had to suffer through some troubles they might not have otherwise experienced. Another aspect of this sale was that the client didn't have any journeyman machinists among their staff. I did provide a fair amount of training in general machining subjects such as fixturing, setup techniques, etc. I doubt other machine tool vendors would have done this sort of thing as they rely on the client already being familiar with machine work. I also wrote enough cnc programs to get them started into production and spent some time teaching them a little programming. Today, they are satisfied with their machine and I'll continue to support them into the foreseeable future. They have hired a machinist to run the machine and have expanded its use to other (milling) operations. I believe there was some sort of short write-up in the December 1998 issue of Design News about their getting this milling machine. During the summer of 1998, I retrofitted an old Wells-Index vertical knee mill for a shop in Millersville, MD. The owner of the shop had bought the machine from a local dealer and had let it sit unused for nearly a year while finishing an addition to their building. When they finally got around to testing out the machine the control died after one day and they never made a single chip! About another year had passed before I saw the machine and got the retrofit job. I was fortuate that I was able to reuse the servomotors, encoders, amps, and a great deal of the relays and associated electrical gear that came with the machine. The job consisted mainly of removing the old paper tape based CPM computer (!) and putting in a PC with the Servo To Go card in it and an Opto22 board to drive the existing relays from the PC's parallel port. The software used was the same as for the Bridgeport. Since this was a job shop, this machine would be used for a variety of applications. Several bugs were discovered while running jobs on this machine. During the process of this implementation Fred came up with a native X Windows user interface called xemc. The main reasons for this were that the operators had difficulty reading the screen using keystick since that was limited to one (small) font. Also, the operation of the control had to be simplified as much as possible since the operators had little computer experience and were not interested in doing a lot of typing. These requirements were satisfied by both the xemc user interface and by the adoption of the K Desktop Environment for X Windows that provides nearly Windows 95 like ease of operation. This conversion took months (into 1999) to bring to completion (not full time of course) and was fraught with many mechanical, electrical, and software problems too numerous to mention and too painful for me to recall. The great bulk of the work on this job was operator training as they felt that there should be a cookbook style set of instructions which would cover any set of circumstances. System reliability got a huge boost during this period since any anomaly in operation was immediately detected by the operators and converted into a show-stopping event. Nevertheless, the job was done and the EMC software acquired a lot of polish in the process. This brings me up to present here in May 1999. (Finally! You must be saying "I though he'd never finish!") At this time the EMC software has been used in several applications and is free enough of bugs to use on a daily basis for actual work. Recently a module to drive stepper motors has been developed and tested although no one has actually used it to run a mill. Also, the java interface problems have been fixed but it hasn't been used in the field. What the project needs to progress further is more users and developers. I have become a big fan of the open source software movement owing mainly to my exposure to Linux and the Internet. The EMC project will be a complete success (IMHO), when major machine tool builders use the software (as is, right from NIST) to control their machines. This won't happen until it's a proven system. That won't happen until there is a large, satisfied user base, and that won't occur unless the risk (translation: cost) is low enough to compel the adventurers to try it. This is what has happened with Linux. At first, only a few technically oriented people tried it because it was free and interesting. These people tested and improved it and plowed their improvements back into the code base until it was a proven system. Today, you can buy a system with Linux preinstalled from quite a few companies (I think Compaq offers this now). It worked for Linux, it can work for the EMC. For more on the philosophy of these ideas you can visit: http://www.opensource.org Be sure to read the essay _The Cathedral and the Bazaar_ by Eric Raymond at: http://www.tuxedo.org/~esr/writings/cathedral-bazaar/ Here is a list of some things that could be worked on: 1. Documentation - The existing documentation lives not only in the doc directory of the distributed software, but also in other electronic documents, e-mails, scraps of paper, and people's brains. This needs to be collected, collated, edited and expanded into a HOWTO, user guide or some other comprehensive reference. I should probably work on this. I owe some time to this project and I'm think I know with where most of the pieces are. 2. Build a stepper machine to see if it works - I'm sure it will and I've got one started now in the form of an old Bridgeport BOSS4. I have gutted it and hope to power the original stepper motors with the Camtronic's 5-amp stepper driver board. 3. Start using and working on the java interface, or write a new GUI using one of the newer, more attractive toolkits like TCL/TK or GTK. 4. Locate or develop a public domain or GPL'ed CAD/CAM system. I really won't be happy until there is a complete end-to-end open source manufacturing system that can exchange data with other popular systems. 5. Add the ability to program the digital I/O with ladder logic like a PLC. The digital I/O is an area that needs customization for almost every machine, especially if you are planning to automate a tool changer. 6. Develop some "open source" hardware - Lots of broken electronics get scrapped because the cost of the time required to figure out how it works enough to troubleshoot the problem exceeds the replacement cost. Often support from the manufacturer is not available, or is priced to encourage buying replacement hardware. With regard to the Servo To Go board it has proven to be a good solution so far, but it may be possible to cut the cost of the capabilities it provides by "cooperative production". I'm not a socialist or communist, I'm just looking to cut the cost of the hardware needed to build cnc machines by reducing the portion of their price that represents intellectual property royalties. 7. Add additional capabilities like rigid tapping, adaptive feed rate control, lathe support, handwheel support (and other pendant functions), probing (both for setup automation and digitizing), leadscrew error compensation tables, etc. I hope that other programmers can write some of this code, rather than starting from scratch with their own cnc projects. Someone (probably Fred) needs to coordinate the satellite development activities and ensure that changes are integrated into the main code distribution. Thanks for reading this far and I hope this provides some food for thought to folks looking to build cnc machines. _______________________________________________ LinuxPLC mailing list [email protected] http://linuxplc.org/mailman/listinfo/linuxplc