Real Time? Was: Shared memory / persistence / io polarity

P

Thread Starter

Phil Covington

I don't necessarily think that this is bad news. As a minimum, if the kernel was patched with UTIME, we could schedule processes with usec
resolution. UTIME is a patch to Linux that increases the temporal granularity of the software clock from its normal 10ms.

The normal linux kernel only checks its list of pending events at each timer interrupt, and therefore, the length of time between interrupts ( a jiffy = 10ms ) is the smallest unit of time with which events can be scheduled. Even if you set your process to be scheduled to SCHED_FIFO (where it gets scheduled in preference to other processes) your process will only get scheduled every 20 ms or so and this will degrade with system loading.

RTLinux is hard real time but you don't have the benefit of the Linux kernel's services. You can communicate to Linux user processes through
FIFOs or shared memory (4MB max that exists outside of the memory that Linux can use for processes). If you want to use Linux's network services, then your driver can't be a RTLinux module. RTLinux is great for driving stepper
motors from the parallel port, as an example. This is what I use it for now.

KU Real Time Linux (KURT) uses UTIME along with providing additional scheduling modes such as SCHED_KURT where KURT real time processes are only
allow to run and SCHED_ALL_PROCS where all processes are allowed to run. All of the Linux kernel's services are available to KURT processes including networking.

If you want to twiddle your I/O down in the usec range and can live without the Linux kernel's services then RTLinux is the only way to go. If you can live with 1 - 10 ms range and need the services of the Linux kernel then KURT is a good choice. If 10 - 30ms is OK then UTIME patch is all you need. The normal Linux kernel will probably be good for 30+ ms range with performance degrading with system load and disk access.

If the LinuxPLC is going to support RTLinux then there are some serious considerations that need to be addressed right now.

Phil Covington
vHMI



----- Original Message -----
From: "Stan Brown" <[email protected]>

> On Mon Jan 17 19:16:29 2000 Phil Covington wrote...
> >
> >From: "Stan Brown" <[email protected]>
> >
> >> On Mon Jan 17 13:08:00 2000 wrote...
> >> >
> >> >In a message dated 1/17/00 9:06:14 AM Pacific Standard Time,
> >[email protected]
> >> >writes:
> >> >> >
> >> >> >My guess is that this approach would be acceptable for the vast majority of
> >> >> >applications. For the few that it is not, we need some way to write the data
> >> >> >table into a faster storage medium (like battery backed RAM) on the fly.
> >> >> >This should not be too difficult to implement.
> >> >>
> >> >> I don't think we can do this, and achieve the speed of scan required for
> >> >> this project to be viable in many applications.
> >> >>
> >> >A couple observations/questions.
> >> >
> >> >1. How fast does the scan need to be to be viable in most applications?
> >100 msec? I'd guess the overwhelming majority of PLC applications do not require
> >> >much more then this.
> >>
> >> Very few of my PLC projects would work with a scan time this slow. Most
> >> I work on require at least a 35MS (faster is better) scan time.
> >>
> >And hence the need for a real time extension to Linux! Linux jiffie resolution is 10ms...
> >If you want scan times under 20-30 ms then you'd better get used to the idea of RTL
> >or KURT or at least a UTIME patch to normal Linux.
>
> Bummer. This is bad news.

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
P

Phil Covington

Here's one more thing I want to add to my last post:

Why is it important to decide whether to support RTLinux right now?

To me, it makes little sense to have real time i/o drivers on the RTLinux side and a userland Logic Engine. The logic engine will not be real time and if it is implemented as a scanned system then there isn't much hope for scan times under 30 ms or so. Realistically it is probably going to be 50 - 100 ms with degradation with system load. And if this is the case, why go through the trouble of writing a RTLinux driver?

KURT or the UTIME patch would improve things on the userland side but is not compatible with RTLinux in the same kernel. There has been some talk about merging the RTLinux and KURT projects, but it seems to have been dropped.

If there is going to be RTLinux support then the Logic Engine should be a RTLinux process also. If we do not support RTLinux then it allows us to use UTIME or KURT and the logic engine and drivers can be KURT processes or at least scheduled as SCHED_FIFO with the UTIME patch in place. People who don't need real time (soft, firm, or hard :)) can use the kernel unpatched. It seems that KURT would give us more choices and leeway in this project. Shared memory can still be used as in the RTLinux case though since it isn't specific to RTLinux.

By the way, has anyone looked at the Shared Memory Driver (mbuff) at http://www.realtimelinux.org ? It was meant for use with RTLinux, but is interesting code to look at for use of shared memory. (The shared memory
that I am discussing here is memory that is set aside that the normal linux kernel can't use for processes; limited to 4MB.)

Comments please?

Phil Covington
vHMI
_______________________________________________

LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
S
I suddenly find myself _very_ concerned about this.

I am not familiar with Linux real time, in any of t's flavors. Could someone who is make a summary of the methods, their pros and cons, and the resultant design limitations (as opposed to standard *NIX programming) which would be imposed by this.

This is worrisome.

Thanks.

--
Stan Brown [email protected] 843-745-3154
Westvaco
Charleston SC.

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
P

Phil Covington

Stan, have you actually read this thread? I am very familiar with the linux kernel's scheduler and most of the real time extensions to linux. I outlined some of the pros and cons of the two popular ones which are RTLinux and KURT. I have used both RTLinux and KURT and have measured the response times of these systems beside the same processes running on the normal linux kernel. The numbers that I have quoted are not
just pulled out of my A**! Sheesh! Go to http://www.realtimelinux.org and do some reading. While you are at it, you may want to look at sched.c in the linux kernel's source
code. After all, unlike a certain much hated Proprietary OS, the the Linux source code is available to all of those who want to understand it.

Phil Covington
vHMI

Ps. There is also ART linux, RTAI, and RED linux in addition to standard
RTLinux and KURT...

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
P

Phil Covington

Phil,

Could you be a little more diplomatic in your replies. We are all trying to read and absorb all of these ideas. It would be very difficult to post an idea if we had to reread and understand every post on the same subject. The list is in the process of being sub-divided into topics, that should make things easier. Until then...

Thanks,

Bill Sturm

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
K
> The silence in this group about this subject concerns me...

It has been mentioned from the very start of the project, but IMHO will require someone who has dealt with it to raise the issues. There are a
few very active posters and coders on the list, and if they come up with a usable prototype, regardless of its limitations or flaws, it will at
least give something to comment on.

I like to recall Fred Brooks, Jr.'s phrase (from "the Mythical Man-Month"), "plan to throw one away" in this context. All is not lost if the project doesn't nail it on the first pass.

I guess what concerns me is the chance that other very similar development has already been done, or is being done in parallel with this one,
by others with similar goals. I've looked around on the LLP (Linux Lab Project), but haven't found quite the same thing, though they deal with a
lot of real IO and control, and have even developed a driver developement scheme (COMEDI) which could be applicable.

--
Ken Irving
Trident Software
[email protected]


_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
P

Phil Covington

The only one that I am aware of is Heinz Haeberle's soft plc code which runs with RTLinux. The code is very interesting to study. It uses a STEP 5 style of programming language. The COMEDI stuff is good code to study also.
NISTs EMC is worth a look too...

Phil Covington
vHMI

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
C
Hi Phil,

Yes. I wish I'd known of Heinz's code earlier. It might have made a good starting point. Although it's a bit premature to say "out of control", I
think we could use a reality check. We are ranging far and afield and going in so many directions at once, I'm not sure how this is going to pull together into a first pass PLC. Quite a few of the posts are good ideas, but neglect basic constraints imposed by programming in the given memory model. But, it's kinda like a dog fight, it's much better to watch than try to participate. I've kinda given up on trying to
compare what is being proposed with what is feasible and prudent to do. Perhaps when the dust clears a little and the signal to moise ratio improves we can repeat the last few points about Linux reality and see if the coders are attempting to deal with them. In the meantime since I have no idea what the interface to my code will look like, I am simply coding on my original plan. Perhaps it will all converge eventually.

Regards

Curt Wuollet,
Wide Open Technologies

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
Actually, no I have not been following this thread very much. I was thinking that we would not need the real time extensions, and that if someone came up with a need later thy could be added.

However it appears that I have been mistaken about this.

While IMHO we really don't need deterministic speed here (which is what I normally think of real time for), we do need speed, speed, and more speed.

You have finally gotten my attention on this with the comment about 19msec scheduling. This is a fair sized chunk of the _entire_
processing speed that we need.

My expertise is in PLC coding, and computers siting on top of them. Not in using the computer for direct control. I am used to needing msec response from my PLC, but only subsecond response from my computer.

This is where I was getting off track.

Having said that, I am trying to get up to speed _fast_ on what the issues with different flavors of Linux real time, and what they imply
for the conceptual model of the software that I had formed.

I apologize for not paying more attention to your earlier postings, but since you clearly do have expertise in this area, could you summarize the choices, and ramifications of each?

Thanks.


--
Stan Brown [email protected] 843-745-3154
Westvaco
Charleston SC.

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
From Phil's prior posts indicating that 30+ msec periodic tasks, using standard linux doesn't seem to be a realistic option for all but the slowest of machine applications. In reading over some of the RTLinux white papers (there's certainly a lot of reading to go through on these real time extensions!) I noticed that Michael Barabanov thesis even mentioned "I found it impossible to reliably run periodic tasks as standard Linux processes, partly because Linux does not provide a periodic timers facility, and also because of other problems described in Chapter 2 <scheduling issues>" in the chapter outlining empirical data for interrupt latency and scheduling precision for
RTLinux versus Linux.

I don't think that a PLC with scan times over 20 msecs for relatively simple ladder programs would be taken seriously in today's market, and I would
think that PID loops should support even the very short time constant processes (say maybe around 200 msec which would require PID loop closure of
20 msec using the 10:1 rule).

On the other hand, without having the support of kernal services if we use RTLinux for the logic engine and I/O drivers, ethernet I/O doesn't seem
practical. I see ethernet I/O as a very important type for this project. Does someone have ideas of how this can be overcome, or do I misunderstand this issue?

From the info in Phil's prior posts it seems like UTIME or KURT options are the most practical available.

Alan Locke
Controls Engineer, Boeing

"My opinions are my own and not necessarily those of my employer"

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
J

Jan Krabbenbos

I can give it a try, but not before sometime next week. I just posted a web address with different ports of Linux, including different attempts
to create real-time versions. I know there has been defined a Posix standard for real-time Unix, and in the past I had a course on a Siemens
Unix implementation of the standard. I need to look around in my archive :) Does somebody know where to get a digital copy of this Posix
standard?

Greetings,
Jan


Jan Krabbenbos e-mail: [email protected]
www : http://www.krabbenbos.com

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
On Wed Jan 19 15:45:43 2000 Jan Krabbenbos wrote...

>Stan Brown wrote:
>>
>> On Mon Jan 17 16:51:21 2000 Curt Wuollet wrote...
>> >
>> >That is the memory model I started the project with to allow the option
>> >for RTL. I'm trying to find out if we're still using that model and not
>> >getting much meaningful response.
>> >
>> I suddenly find myself _very_ concerned about this.
>>
>> I am not familiar with Linux real time, in any of t's flavors. Could
>> someone who is make a summary of the methods, their pros and cons, and
>> the resultant design limitations (as opposed to standard *NIX programming)
>> which would be imposed by this.
>
>I can give it a try, but not before sometime next week. ...<clip>

OK, you have touched on one of my real concerns here. If we have to have a real time extension, just to make the base unit work, then we are tied to that extension.

I _really_ wanted to avoid that, so that this project would be portable to any flavor of *NIX. Much as I like Linux, there are other flavors that I would like this to run on. FreeBSD, and HP-UX for instance.

--
Stan Brown [email protected] 843-745-3154
Westvaco
Charleston SC.

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
C
Stan Brown wrote:

> OK, you have touched on one of my real concerns here. If we have to
> have a real time extension, just to make the base unit work, then we
> are tied to that extension.

From what I read last night, KURT is portable to any platform that supports utime. Since, if I recall, that was a BSD idea in the first place, I'm fairly sure it would be portable to *BSD. I wouldn't venture a guess on HP-UX but doing this kind of work on any closed, proprietary system is going to be difficult. Perhaps HP might be interested, but, I doubt it. A bigger concern is that we would want to include KURT in the source
tree and maintain it for our purposes. We pretty much need kernel hacker type talent to support it and help out. It seems non-trivial to use. Of course, PLC internals are non-trival also.


_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
So we would find ourselves in the bussiness of keepping our own version of KURT? I don't quite see why wee would need to do that.

Could you elaborate please?

Sounds a little better on the portabilty front than I first thought. Anyone know if indeed KURT has bee ported to {Freee|Open|Net}BSD ?

--
Stan Brown [email protected] 843-745-3154
Westvaco
Charleston SC.
_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
C
Stan Brown wrote:
>
> On Thu Jan 20 17:38:58 2000 Curt Wuollet wrote...
> >
> >>From what I read last night, KURT is portable to any platform that supports > >utime. Since, if I recall, that was a BSD idea in the first place, I'm fairly > >sure it would be portable to *BSD. I wouldn't venture a guess on HP-UX > >but doing this kind of work on any closed, proprietary system is going to > >be difficult. Perhaps HP might be interested, but, I doubt it. > >A bigger concern is that we would want to include KURT in the source > >tree and maintain it for our purposes. We pretty much need kernel hacker
> >type talent to support it and help out. It seems non-trivial to use. Of course, > >PLC internals are non-trival also.
>
> So we would find oureselves in the bussiness of keepping our own version of KURT? I don't quite see why wee would need to do that.
>
> Could you elaborate please?

We would want source control on the version that we use so that everyone can stay in sync until any new versions are tested and any needed changes
(extra parms, etc.) in our code are complete and tested. That way we could avoid upgrades unless they provide needed functionality. Also, who knows, Kansas University could decide to commercialize it. Speaking of that, I forgot to check the licensing. It's not an option if it's not GPL. People who really understand it could help integrate it and perhaps add any special stuff for our situation. Playing with the Linux kernel is not for the weak at heart, the subtleties are legion and it's easy to introduce
bugs. One little memory leak and your uptime decreases dramatically.

cww

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
Curt Wuollet wrote:
>Also, who knows, Kansas University could decide to commercialize it. Speaking
>of that, I forgot to check the licensing. It's not an option if it's not GPL.

From the KURT 2.0 readme:

Copyright and Credits

Copyright (C) 1997-1999 by the University of Kansas Center for Research, Inc. This software was developed by the Information and
Telecommunication Technology Center (ITTC) at the University of Kansas. Partial funding for this project was provided by Sprint. This software may be used and distributed according to the terms of the GNU Public License, incorporated herein by reference. Neither ITTC nor Sprint accept any liability whatsoever for this product.



_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
C
Yes, I did some more reading last night. As often happens it raised more questions than answers. I need to study it more but, a nagging concern
was raised. We are wanting to schedule tasks that take appreciable time to complete. We will probably have several. IF we switch to RT mode, will we ever get switched back to run "normal" tasks. If all resources are devoted to RT, in our case, that won't neccesarily speed things up.
Fieldbus IO, for example will take it's sweet time regardless of how many machine cycles we make available. IF these overlap due to external
causes, we may not leave RT mode. Slow serial I/O is not well suited for this model. We may need an interesting mix of RT tasks for fast IO and interrupt driven processes for the rest. The interaction should prove interesting. Traditionally this is met with a top end that
executes quickly and a bottom end that waits for the result. It would help to get an idea of the turn around time of the various type of I/O supported. Does anybody know enough about it yet to put these concerns to rest?

Curt W
Wide Open Technologies

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc
 
Top