# io library and file parsing

M

#### Mario de Sousa

Jiri and all,

Now that I have written a couple of io modules I think I have started to understand the issues involved. I would say that almost 50% of the
effort goes into parsing the config file for the io mapping parameters, i.e. the io mapping table.

Issue 1
-------

Is there any way we could abstract this away and make it common to all io modules? This would also make all io modules more coherent when it
comes to configuring them... For example, I added an invert option to the parport module that tells the module to invert the linuxplc point before copying to the parallel port (or vice-versa). I think all io modules should probably also support this. This is an example of what could be placed in the common library. Maybe we could even write a
complete abstract io module, with some callbacks to:
init()
this would read and parse module specific config data, such as io base address, IP address, etc...
parse_point()
this would parse the module specific parameters to to a specific point, this would be called for every line in the io mapping table...
read a point from physical io
write_point()
write a point to physical io
close()
any cleanup that the module may have to do before terminating.

To write a module, the author would only have to write the above five functions, and presto, the module was ready to go. Like this he could
concentrate on the specific physical io details, and not have to spend time re-doing all the boring stuff related to interfacing with the
linuxplc.

Maybe this would not be enough for every type of io module, but for those for which it isn't, then the module can always be coded from scratch, using the linuxPLC library API.

|--------------------|---------|
| IO module 1 | io 2 | io |
|--------------------| module |
| Generic io module | 3 |
| API | |
|------------------------------|
| linuxPLC library API |
|------------------------------|

Issue 2
-------

For logic type modules, these will have many scratch variables that maybe don't need to be interfaced to any other module. This raises the
'philosophical' question of whether these variables should be mapped onto linuxplc points.

Take the iec compiler for example. We can take the route of (a) having every Mx.y 'variable' be a linuxplc point, which means they all have to
be configured, or (b) we can choose to assume that this memory is private to the iec logic module, and only explicitly configured memory
locations would be mapped onto linuxplc points.

Which is the correct way to go? I think we really must discuss this now. It will influence how the memory manager (a.k.a. the smm, which
actually is now called the gmm in the library) will be coded, and what types of linuxplc points will be supported.

I wanted to start tackling online configuration changes, but I feel we should lay the ground rules for this first, before charging into
something that would probably have to be changed later.

Just throwing some ideas around... ;-)

Cheers,

Mario.

--
----------------------------------------------------------------------------
Mario J. R. de Sousa
[email protected]
----------------------------------------------------------------------------

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

J

#### Jiri Baum

Hello,

(I'm answering to the two issues separately)

Mario de Sousa:
> I would say that almost 50% of the effort goes into parsing the config
> file for the io mapping parameters, i.e. the io mapping table.

> Issue 1
> -------

> Is there any way we could abstract this away and make it common to all io
> modules?

We should be able to.

> This would also make all io modules more coherent when it comes to
> configuring them... For example, I added an invert option to the parport
> module that tells the module to invert the linuxplc point before copying
> to the parallel port (or vice-versa). I think all io modules should
> probably also support this.

Yes, this should be part a general scaling facility. Presumably it'll initially offer just invert, but eventually it should support all sorts of goodies

> Maybe we could even write a complete abstract io module, with some
> callbacks to:
...
> parse_point()
> this would parse the module specific parameters to to a specific
> point, this would be called for every line in the io mapping table...

To some extent, this is part of what should be abstracted.

> read a point from physical io

Would this really be convenient? Most io modules, I'd think, receive a chunk of data from physical io which might belong into several separate
lPLC points - eg, receive a byte which corresponds to 8 digital inputs.

> Maybe this would not be enough for every type of io module, but for those
> for which it isn't, then the module can always be coded from scratch,
> using the linuxPLC library API.

Definitely.

Jiri
--
Jiri Baum <[email protected]>
You know you've been hacking too long when ...
... reading a book you notice the word "From" at the beginning of a line.

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

M

#### Mario de Sousa

Jiri Baum wrote:
> > Issue 1
> > -------
>
> > Is there any way we could abstract this away and make it common to all io
> > modules?
>
> We should be able to.
>
> > This would also make all io modules more coherent when it comes to
> > configuring them... (...)
>
> Yes, this should be part a general scaling facility. Presumably it'll
> initially offer just invert, but eventually it should support all sorts of
> goodies
>
> > Maybe we could even write a complete abstract io module, with some
> > callbacks to:
> ...
> Would this really be convenient? Most io modules, I'd think, receive a
> chunk of data from physical io which might belong into several separate
> lPLC points - eg, receive a byte which corresponds to 8 digital inputs.
>

Yes, I changed my mind myself when I started to think this through over the weekend for exactly the same reason.

Here is my second take on this.

plc_io_parse_map()
parses the io map. It stores the map into a list/array/whatever of struct(s) that are defined in a plc_io.h file, to be included by modules
using this library. This structure must contain a void * for private data defined by the io module. The io module can store module dependent
configuration data using this pointer. This function does a callback to the io module so it can parse the module dependent data.
Having the list/array/whatever structure already defined saves some boring repetitive coding. We would then have a collection of functions to access the list values.

Let the io module have it's own main loop. This main loop would call
plc_io_get()
plc_io_set()

instead of the lower level plc_get() and plc_set(). The above two functions would then handle the inversion, scaling, etc... Like this, if we ever decide to change or augment these facilities, we only have to do it once.

Does this make sense to you?

Mario.

--
----------------------------------------------------------------------------
Mario J. R. de Sousa
[email protected]
----------------------------------------------------------------------------

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

J

#### Jiri Baum

Mario de Sousa:
> > > Maybe we could even write a complete abstract io module, with some
> > > callbacks to:
> > ...
Jiri Baum:
> > Would this really be convenient? Most io modules, I'd think, receive a
> > chunk of data from physical io which might belong into several separate
> > lPLC points - eg, receive a byte which corresponds to 8 digital inputs.

Mario de Sousa:
> Here is my second take on this.

> plc_io_parse_map() parses the io map. It stores the map into a
> list/array/whatever of struct(s) that are defined in a plc_io.h file, to
> be included by modules using this library. This structure must contain a
> void * for private data defined by the io module.

I'd rather avoid the void *.

It'd be better to allow custom fields; this would be a pain to do/use as a library, but fairly easy if it's a pre-processor kind of thing. Give it a
table spec and it spits out a parser fn. It's not even (necessarily) I/O.

> This function does a callback to the io module so it can parse the module
> dependent data.

While this is certainly necessary, I'd prefer if its usage could be rare. So that the module-dependent data could be parsed uniformly, so far as possible.

> Let the io module have it's own main loop. This main loop would call
> plc_io_get()
> plc_io_set()

> instead of the lower level plc_get() and plc_set().

Yes. Also, plc_io_[gs]et() would optionally be byte/word oriented for digital i/o. That way:
- the module doesn't have to worry about splitting up bits, and
- multi-bit points can be handled (where several adjacent bits form
a binary/BCD/gray value)

> The above two functions would then handle the inversion, scaling, etc...
> Like this, if we ever decide to change or augment these facilities, we
> only have to do it once.

Yes. We might want to:
invert
reverse
binary/BCD/gray
scale
linearize (generic and/or thermocouple)

Jiri
--
Jiri Baum <[email protected]>
You know you've been hacking too long when ...
... reading a book you notice the word "From" at the beginning of a line.

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

C

#### Curt Wuollet

Jiri Baum wrote:
>
> Mario de Sousa:
> > > > Maybe we could even write a complete abstract io module, with some
> > > > callbacks to:
> > > ...
> Jiri Baum:
> > > Would this really be convenient? Most io modules, I'd think, receive a
> > > chunk of data from physical io which might belong into several separate
> > > lPLC points - eg, receive a byte which corresponds to 8 digital inputs.
>
> Mario de Sousa:
> > Here is my second take on this.
>
> > plc_io_parse_map() parses the io map. It stores the map into a
> > list/array/whatever of struct(s) that are defined in a plc_io.h file, to
> > be included by modules using this library. This structure must contain a
> > void * for private data defined by the io module.
>
> I'd rather avoid the void *.
>
> It'd be better to allow custom fields; this would be a pain to do/use as a
> library, but fairly easy if it's a pre-processor kind of thing. Give it a
> table spec and it spits out a parser fn. It's not even (necessarily) I/O.
>
> > This function does a callback to the io module so it can parse the module
> > dependent data.
>
> While this is certainly necessary, I'd prefer if its usage could be rare.
> So that the module-dependent data could be parsed uniformly, so far as
> possible.
>
> > Let the io module have it's own main loop. This main loop would call
> > plc_io_get()
> > plc_io_set()
>
> > instead of the lower level plc_get() and plc_set().
>
> Yes. Also, plc_io_[gs]et() would optionally be byte/word oriented for
> digital i/o. That way:
> - the module doesn't have to worry about splitting up bits, and
> - multi-bit points can be handled (where several adjacent bits form
> a binary/BCD/gray value)

Hooray! cww

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

M

#### Mario de Sousa

Jiri Baum wrote:
>
> Mario de Sousa:
> > > > Maybe we could even write a complete abstract io module, with some
> > > > callbacks to:
> > > ...
> Jiri Baum:
> > > Would this really be convenient? Most io modules, I'd think, receive a
> > > chunk of data from physical io which might belong into several separate
> > > lPLC points - eg, receive a byte which corresponds to 8 digital inputs.
>
> Mario de Sousa:
> > Here is my second take on this.
>
> > plc_io_parse_map() parses the io map. It stores the map into a
> > list/array/whatever of struct(s) that are defined in a plc_io.h file, to
> > be included by modules using this library. This structure must contain a
> > void * for private data defined by the io module.
>
> I'd rather avoid the void *.
>
> It'd be better to allow custom fields; this would be a pain to do/use as a
> library, but fairly easy if it's a pre-processor kind of thing. Give it a
> table spec and it spits out a parser fn. It's not even (necessarily) I/O.
>

OK. I see two levels of functions arising from this discussion.

1) The first level is a generic function for the conffile.h interface:
What I suggest for this is an sscanf () style function that would accept
backus-naur (is this correctly spelled?) type of formating. This would be a generic
conffile_xxxxxx(const char *table_name, const char *format, ...)
function with variable number of arguments (just like the scanf() functions).
This would accept format strings like:
"[inv] {C | D | S} (0..7)"
Actually, this would have to be adulterated to specify the type of pointer in which to store the value being parsed, so it would probably be something like:
conffile_xxxx("table_name_1", "[inv]%s {C | D | S}%c (0..7)%d", char **,
char *, int *)
Offcourse, this means we can't use the '{', '}', '[', ']', '(' and ')'
characters for the configuration, unless we accept escaped characters, which would complicate things for a first round implementation.

2) The second level would be io module specific. We have a plc_io_parse_map() function that will populate an io mapping list/array/... Some of the info will be generic, relating to bit
inversion/reversal/scaling/..., other info will need to be module specific.
For the module specific data, two solutions seem to on the table:
a) callback function, using void *, with the called backed function having to malloc the required memory.
b) automatically generated function at compile time. Jiri, you are the Perl enthusiast/guru around here. If we take this option, you will
probably have to write this yourself... Could you be a little more forthcoming on what you are proposing for this? What kind of interface would this support?

----------------------------------------------------------------------------
Mario J. R. de Sousa
[email protected]
----------------------------------------------------------------------------

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

J

#### Jiri Baum

Jiri Baum wrote:
> > I'd rather avoid the void *.

> > It'd be better to allow custom fields; this would be a pain to do/use
> > as a library, but fairly easy if it's a pre-processor kind of thing.
> > Give it a table spec and it spits out a parser fn. It's not even
> > (necessarily) I/O.

Mario de Sousa:
> OK. I see two levels of functions arising from this discussion.

> 1) The first level is a generic function for the conffile.h interface:
> What I sugest for this is an sscanf () style function that would accept
> backus-naur (is this correctly spelled?) type of formating.

The disadvantage being that it doesn't do type-checking. With a preprocessor, gcc then goes through and checks all the types (and optimizes
whatever might be optimized, though that's not so important).

> Actually, this would have to be adulterated to specify the type of
> pointer in which to store the value being parsed, so it would probably be
> something like:
> conffile_xxxx("table_name_1", "[inv]%s {C | D | S}%c (0..7)%d", char **,
> char *, int *)

An optional "inv" keyword should be returned by the library as a boolean. Similarly, a C|D|S should be returned as an enumerated type, preferably allowing #define'd constants.

> b) automatically generated function at compile time. Jiri, you are the
> Perl enthusiast/guru around here. If we take this option, you will
> probably have to write this yourself... Could you be a little more
> forthcoming on what you are proposing for this? What kind of interface
> would this support?

In the .h file, you would have:

typedef struct {
plc_io_pt pt;
u8 opts;
} point_info;

And in the table config file you might have (zeroth-draft syntax!):

table map # parse the "map" table
iopoint pt # it begins with an iopoint spec,
# to go into field pt
option foo opts bit FOO_OPTION # keyword foo -> opts |= FOO_OPTION
option bar opts bit BAR_OPTION # keyword bar -> opts |= BAR_OPTION

Of course, this is not at all thought through at this point; there are some things I already know would be interesting, and no doubt there are others I haven't thought of yet.

It could be implemented in Perl, or in lex/yacc (flex/bison). Both of them have their advantages and disadvantages (Perl does better strings, bison
does better parsing).

Jiri
--
Jiri Baum <[email protected]>
You know you've been hacking too long when ...
... reading a book you notice the word "From" at the beginning of a line.

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

M

#### Mario de Sousa

Jiri Baum wrote:
>
> Jiri Baum wrote:
> > > I'd rather avoid the void *.
>
> > > It'd be better to allow custom fields; this would be a pain to do/use
> > > as a library, but fairly easy if it's a pre-processor kind of thing.
> > > Give it a table spec and it spits out a parser fn. It's not even
> > > (necessarily) I/O.
>
> Mario de Sousa:
> > OK. I see two levels of functions arising from this discussion.
>
> > 1) The first level is a generic function for the conffile.h interface:
> > What I sugest for this is an sscanf () style function that would accept
> > backus-naur (is this correctly spelled?) type of formating.
>
> The disadvantage being that it doesn't do type-checking. With a
> preprocessor, gcc then goes through and checks all the types (and optimizes
> whatever might be optimized, though that's not so important).
>

It would probably still be useful? It could be called by the automatically generated parsing function which does the typechecking?

> > Actually, this would have to be adulterated to specify the type of
> > pointer in which to store the value being parsed, so it would probably be
> > something like:
> > conffile_xxxx("table_name_1", "[inv]%s {C | D | S}%c (0..7)%d", char **,
> > char *, int *)
>
> An optional "inv" keyword should be returned by the library as a boolean.
> Similarly, a C|D|S should be returned as an enumerated type, preferably
> allowing #define'd constants.

Yes... I was just trying to get the general idea across without complicating it too much. ;-)

> > b) automatically generated function at compile time. Jiri, you are the
> > Perl enthusiast/guru around here. If we take this option, you will
> > probably have to write this yourself... Could you be a little more
> > forthcoming on what you are proposing for this? What kind of interface
> > would this support?
>
> In the .h file, you would have:
>
> typedef struct {
> plc_io_pt pt;
> u8 opts;
> } point_info;
>
> And in the table config file you might have (zeroth-draft syntax!):
>
> table map # parse the "map" table
> iopoint pt # it begins with an iopoint spec,
> # to go into field pt
> (...)

OK. Now I see where you are going. Anybody want to implement this?

We still have to nail down the interface for the
plc_io_get()
plc_io_set()

iterating through the mapped points list, etc...

and the
plc_io_parse()

I don't have much time right now. I'll hopefully get back to this later on in the day.

Mario.

----------------------------------------------------------------------------
Mario J. R. de Sousa
[email protected]
----------------------------------------------------------------------------

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

J

#### Jiri Baum

Jiri Baum (about a sscanf()-style function):
> > The disadvantage being that it doesn't do type-checking. With a
> > preprocessor, gcc then goes through and checks all the types (and
> > optimizes whatever might be optimized, though that's not so important).

Mario de Sousa:
> It would probably still be useful? It could be called by the
> automatically generated parsing function which does the typechecking?

Once you have an automatically-generated parsing function, it's easier (and more efficient) to call a type-specific parsing function (or parse it on the spot).

> > > b) automatically generated function at compile time.
...
> OK. Now I see where you are going. Anybody want to implement this?

Heh, that's the trick, isn't it...

I shouldn't be taking the time to do it right now, but I'll see how it goes. It might be that I'll write at least a simple one.

Question: Perl or flex/bison?

> We still have to nail down the interface for the
> plc_io_get()
> plc_io_set()

> iterating through the mapped points list, etc...

That's another question: if we allow plc_io_[gs]et() to be byte-oriented, there's no longer a 1-to-1 correspondence between these and lPLC points.

This is another one of those hide a lot of complexity away'' interfaces. We want both the conffile syntax and the API to be simple, and we'll have to think of something clever to go between them.

Iterating through the list won't be a problem; the parsing function will return either a linked list or an array, iterate as you wish. The
translator could be written to allow both, but I don't see the point.

Jiri
--
Jiri Baum <[email protected]>
You know you've been hacking too long when ...
... reading a book you notice the word "From" at the beginning of a line.

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

M

#### Mario de Sousa

Jiri Baum wrote:
>
> Jiri Baum (about a sscanf()-style function):
> > > The disadvantage being that it doesn't do type-checking. With a
> > > preprocessor, gcc then goes through and checks all the types (and
> > > optimizes whatever might be optimized, though that's not so important).
>
> Mario de Sousa:
> > It would probably still be useful? It could be called by the
> > automatically generated parsing function which does the typechecking?
>
> Once you have an automatically-generated parsing function, it's easier (and
> more efficient) to call a type-specific parsing function (or parse it on
> the spot).

Yes, I was thinking on the possible reduction in compiled code size. But no problem, we can worry about/implement that later if it starts to become an issue.

> > > > b) automatically generated function at compile time.
> ...
> > OK. Now I see where you are going. Anybody want to implement this?
>
> Heh, that's the trick, isn't it...
>
> I shouldn't be taking the time to do it right now, but I'll see how it
> goes. It might be that I'll write at least a simple one.
>
> Question: Perl or flex/bison?

I would prefer flex/bison, for a very simple but personal reason; I have never needed to learn either Perl or flex/bison, and as the iec
compiler is being written in flex/bison, like this I will be able to understand both libraries by learning only one of the above.

If you really think that there is no technical advantage to go with either, would you be willing to give my request some consideration?

> > We still have to nail down the interface for the
> > plc_io_get()
> > plc_io_set()
>
> > iterating through the mapped points list, etc...
>
> That's another question: if we allow plc_io_[gs]et() to be byte-oriented,
> there's no longer a 1-to-1 correspondence between these and lPLC points.
>

Correct. I was thinking of keeping it lPLC point oriented. The io module would have to iterate through the whole list and build up the byte in intermediate (local/private) variables, before writing to physical io. I think we should take this one step at a time. Later on, if we see some more patterns forming we can add it to the library, but for now I think we should try to keep it simple.

> This is another one of those hide a lot of complexity away'' interfaces.
> We want both the conffile syntax and the API to be simple, and we'll have
> to think of something clever to go between them.

Like I said above, I think we should leave the hiding of complexity to a later stage. For now, just try and make life simpler when it comes to
parsing the config file, massaging the lPLC point data (inverting/reversing/scaling/...), and managing the structures required to store all this info.
In other words, not necessarilly reduce the complexity of the interface, but more importantly reduce the tedium of writing the repetitive code of an io module.

> Iterating through the list won't be a problem; the parsing function will
> return either a linked list or an array, iterate as you wish. The
> translator could be written to allow both, but I don't see the point.

Please note that it is not only the translator that needs to iterate through the list. The io module will need to iterate through it to get the current (massaged) state of each io mapped lPLC point.

Yes, agreed, the iteration functions is easy. Nevertheless I think we should try and isolate the io modules from the internal structure the
library will be using to store the info, so we may later decide to change the way the library works internally without having to change any io module.

Mario.

----------------------------------------------------------------------------
Mario J. R. de Sousa
[email protected]
----------------------------------------------------------------------------

_______________________________________________
LinuxPLC mailing list
[email protected]
http://linuxplc.org/mailman/listinfo/linuxplc

J

#### Jiri Baum

> > Jiri Baum (about a sscanf()-style function): > > > > The disadvantage being that it doesn't do type-checking. With a > > > > preprocessor, gcc then goes through and checks all the types (and > > > > optimizes whatever might be optimized, though that's not so important). > > Mario de Sousa: > > > It would probably still be useful? It could be called by the > > > automatically generated parsing function which does the typechecking? Jiri Baum: > > Once you have an automatically-generated parsing function, it's easier > > (and more efficient) to call a type-specific parsing function (or parse > > it on the spot). Mario de Sousa: > Yes, I was thinking on the possible reduction in compiled code size. > But no problem, we can worry about/implement that later if it starts to > become an issue. No problem, calling a type-specific function doesn't take any more space than calling a generic one, and the function itself can be in the shared library (and, again, won't be bigger than the corresponding section of the generic function). [... other points snipped here - should return to them later ...] > > Iterating through the list won't be a problem; the parsing function > > will return either a linked list or an array, iterate as you wish. The > > translator could be written to allow both, but I don't see the point. > Please note that it is not only the translator that needs to iterate > through the list. The io module will need to iterate through it to get > the current (massaged) state of each io mapped lPLC point. Yes - but both implementations are sufficiently easy that it almost doesn't matter. > Yes, agreed, the iteration functions is easy. Nevertheless I think we > should try and isolate the io modules from the internal structure the > library will be using to store the info, so we may later decide to > change the way the library works internally without having to change any > io module. True, but then we'd need to provide iterator functions. Not a problem, but the question of the interface remains. Jiri -- Jiri Baum <[email protected]> I can't figure out whether keeping all my friends in a tickle file would be horribly impersonal or over-familiar. _______________________________________________ LinuxPLC mailing list [email protected] http://linuxplc.org/mailman/listinfo/linuxplc