using int32 to store everything from uint8 no int32 with values below 2 billion - Page 1
Forum section: electronics / general

author: Simon
author_rank: Unknown
timestamp: February 23, 2024, 12:31:12 pm
content: So I am working with CAN Open and the faff is that I need to store and send values in 8, 16 and 32 bit signed or unsigned. This makes it impossible to say put them all in an array that would be very useful for say stepping through with easy to write code to send the setup values one after the other.but it occurs to me. with all the memory I am already using to store pointers to variables that hold the relevant value I am already using 32 bits of memory anyway. And fromwhat I understand of two's complement representation a negative 8 or 16 bit number stored in a signed 32 bit space looks the same as when stored in an 8 or 16 bit space once the 2 or 3 leading bytes are removed.Therefore providing where I have an unsigned 32 bit value I do not need to go above the halfway mark of a little over 2 billion then sending the number as a signed or unsigned when I represent an unsigned number in the signed space it makes no difference. This means that I can now have an array of signed 32 bit values that will hold any data type I am using. as I use memcpy() to move the data about the actual storage type is irrelevant. I know what size each variable is anyway as I have to store the information about the data size along with my record of the object dictionary (also in an array), so bingo.This makes it easier to setup devices when I have to send a sequence of values as I can now while index is has not reached this value ect. In my actual program everything is a signed 32 bit number and the world is a better place for me

author: brucehoult
author_rank: Super Contributor
timestamp: February 23, 2024, 02:25:02 pm
content: Umm .. this is more a comment than a question ... but ...

author: Simon
author_rank: Unknown
timestamp: February 23, 2024, 04:02:12 pm
content: It is a question. To be honest to me it looks like it is made to be so. In canopen land as part of your configuration you have to send the index (16 bits), subindex (8 bits) and number of bits (8 bits) to the devices. so I already have 32 bit values that define the complete "location" of the data and the length. But there is no way to identify signed or unsigned. But if I just store everything as 32 bit signed then the bit length field specifies how much of the 4 bytes to send.-2 is 0x FF FF FF FE in 32 bit-2 is 0x FF FE in 16 bitso if I need to use this value as 16 bit I can take the 32 bit value and truncate it. Providing I do not have a positive number that is so high it breaks into the negative range a positive signed 32 bit number is the same as an unsigned 32 bit number, 32 bits signed allows me to store 2'147'483'648. The most granular number I have come across so far is mA to define my motor current. I really don't think I will be using a motor that runs on 2'147'483 A (2 MA). Same for voltage in mV. even µV.So am I nuts? or was this all meant to be?

author: mariush
author_rank: Super Contributor
timestamp: February 23, 2024, 04:39:59 pm
content: You could probably have a separate array of bytes where you use 3 or 4 bits for every number : 2 bits or 3 bits for size (1 byte, 2 byte, 3 byte , 4 byte, more if you use 3 bits) and one for signed/unsignedThen store the numbers as a series of bytes. If you want to parse the numbers, you just keep reading 4 bits at a time from the first array, then read up to n bytes from the second array and advance position in both arrays.If you want to get more fancy, you could use a 32 bit value for every (up to) 8 numbers : 4 bits for the number of values, 4 bits for whatever and 24 bits for the 8 sets of 3 bits (data length and sign) ... or 4 bits for number of values, 28 bits for 7 x 4 bitsif you use uint32 for everything 8 numbers would use 8 x 4 = 32 bytes. with the idea above, 8 uint8 numbers could use 4 bytes + 8 x 1 byte = 12 bytes.

author: Simon
author_rank: Unknown
timestamp: February 23, 2024, 05:01:12 pm
content: Well you see the problem is that I need to be able to cycle through an array. At the moment I have an array of pointers to the variables. So if I had say at array of single bytes I don't see how I would know the difference between data and the byte/signedness of the data.So I have to send various variables to devices, at the moment I can cycle through them if I rearrange their pointers in my pointer array, but if I have the pointer array then I may as well use those 4 bytes to store the data, the problem is getting at different data types as though in an array. There is a parallel array that carrys the standard locations like 0x60410010, the index of these is the same as the index for my array of pointers. It means that first I have to setup a variable and assign it a value, then put a pointer t this variable into the pointers array. As all pointers are the same type the array works.But if I just call everything int32 then the individual variables go away and free up memory and I just use the values at the locations and save pointery stuff.

author: eutectique
author_rank: Frequent Contributor
timestamp: February 23, 2024, 05:06:06 pm
content: Or you can use a TLV thing, which is type-length-value. The generic entity is declared asCode:typedef struct {uint8_t type;uint8_t length;uint8_t value[];} tlv_t;typeis an enum of int8, uint8, int16, uint16, MAC48, whatever.lengthis the length ofvaluein bytesvalueis the payload

author: Colt45
author_rank: Contributor
timestamp: February 23, 2024, 05:24:33 pm
content: Yes, better to keep the payload as 8bit array in a struct.

author: Simon
author_rank: Unknown
timestamp: February 23, 2024, 06:06:53 pm
content: The array of bytes cannot be used as is as a value. This is not about generating the can message payload, it is about storing the data locally. So currently there is:uint32 OD_entry["i"] this is the list of object dictionary items.void * OD_value["i"] This contains the pointers to the variables of 8, 16 or 32 bit signed or unsigned."i" is replaced by definitions of the object name so for example#define STATUS_WORD 0x03This way the name of the object can be used instead of the number.At the start of the program I have to assign the dictionary location and bit count to the OD_entry array and the pointer to the variable to OD_value.So for exampleuint16 STATUS_WORD_val ;So:OD_entry[STATUS_WORD ] = 0x60400010 ; // index 0x6040, subindex 0x00, number of bits 16.OD_value[STATUS_WORD ] = &STATUS_WORD_val ;So now I can use the object name to index the array of its definition or it's value. Obviously I can only memcpy() to and from the pointer but I have access to the variable.This means that when I receive a SDO message which contains the index, sub index and data length I can reconstruct the definition and find it in the OD_entry array, when I find a match I know which index of the OD_value array will hold the pointer to the variable.So lots of messing about and up to 12 bytes per value, most are 10. So if instead I use int32 variables all round I can put them all into one OD_value array that now is not of the void * type but just int32. This means that I can use the values directly and 8 bytes will be used all round.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 24, 2024, 11:54:15 am
content: I don't like language lawyerism, but since we are talking about integer casts, perhaps some consideration for the C standard is appropriate here.TL;DR: Your approach works fine.The key part is in C99-C17 6.3.1.3:Quote from: ISO C Standard, 6.3.1.3This means that conversion from signed to unsigned uses modular arithmetic, and conversion to a signed integer type is exact and lossless as long as the value is representable in the target signed type; otherwise it is implementation-defined. (GCC and Clang both support modular rules for signed integer target types also, but may have (now in the future) options to change this.)The other important thing is what the standard says about exact-width integer types (intN_t,uintN_t), in C99 7.18.1.1 and C11-C17 7.20.1.1:Quote from: ISO C standard, 7.18/20.1.1It goes on to say that the types forN= 8, 16, 32, 64 are optional, but if supported, these types (typedef'd) will be supported. For practical purposes, all widely-used C and C++ compilers do now support these (synthesizing them even when the hardware does not natively support them) from tiny C compilers to flagship commercial C++ compilers, so we can basically assume they (and the related least-width and fast types) are supported, possibly excepting theN= 64 when dealing with 8-bit targets and older compiler versions.Thus, based on the above observations from the text of the standard:While there may be standards-compliant C and C++ compilers that do not support these types (<stdint.h>/<inttypes.h>are provided by the compiler, not by the standard C library per se;), they are completely impractical, and can be discounted. On all architectures, the compilers can synthesize the support for these types.When supported, it is absolutely safe to use anint32_tvariable to hold any unsigned value between 0 and 232-1 = 4294967295, inclusive; and any signed value between -231= -2147483648 and 231-1 = 2147483647, inclusive; as long as you know whether the value is supposed to be signed or unsigned. The cast to the target type will never fail, or produce an incorrect value.

author: Simon
author_rank: Unknown
timestamp: February 24, 2024, 10:37:14 pm
content: Quote from: Nominal Animal on February 24, 2024, 11:54:15 amBut this is the main crux of the problem. When I receive data I know how many bytes the variable is, but I do not know if it is signed. So dumping everything into a signed 32 bit integer kind of simplifies it as I know that i will never use values above 2 billion.I guess this still raises problems as I do not know if a 16 bit value is signed or not, if I cast a high 16 bit unsigned value to 32 bit as though signed I have a problem. i can't distinguish, if I simply pad out a negative 16 bit value with 16 bits set to 0 I now generate an incorrect positive 32 bit number.OK, so all is not lost. I will have to update my Object definition to include a sign marker.I was using the same format CAN Open wants me to use to send object definitions when I set up tho PDO stuff, but this is a small part of using CAN Open so converting back to this format is not a problem. I will need to change 0x60400010 to something like 0x60400090 so just add 0x80 for signed numbers. I can then put the number into a signed or unsigned holding value.When I send I simply truncate any int32 down to the size of the variable.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 25, 2024, 01:44:42 am
content: What do you need such a generic associative array for CANopen, anyway?For 11-bit CAN-IDs, there are 2048 possible IDs. 0x000 (NMT), 0x580–0x67F (SDO), 0x700–0x77F (guard/heartbeat), and 0x7E4–0x7E5 (layer setting services) are reserved and fixed, and only 1600 or so (0x001–0x580, 0x680–0x6DF, 0x780–0x7DF) are really available for PDOs (default four for receiving are 0x200+nodeper object dictionary key 0x1400, 0x300+nodeper key 0x1401, 0x400+nodeper key 0x1402, and 0x600+nodeper key 0x1403; default four for transmitting are 0x180+nodeper key 0x1800, 0x280+nodeper key 0x1801, 0x380+nodeper key 0x1802, and 0x480+nodeper key 0x1803), SYNC (default 0x080 per key 0x1005), TIME (default 0x100 per key 0x1012), and EMCY (default 0x080+nodeper key 0x1014) messages.Thus, initially, a slave only listens for less than a dozen CAN-IDs, with the rest configured dynamically. For the receive side, I'd use an array containing at least the CAN-ID and a function pointer to the message handler, sorted by decreasing CAN ID, so a binary search will locate the correct message handler very, very fast, without wasting memory. (Adding a new receive CAN-ID is then an insertion operation, moving any entries at or after the insertion point one entry outwards; and removing a CAN-ID is a corresponding delete operation, moving any entries at or after the insertion point one entry inwards.)The object dictionary index 0x0000 is never used, so can be used as an "unused" or "end of list" sentinel.Indexes 0x0001–0x001F define fixed data types (code DEFTYPE), and optionally, when read yield the length (in bits) of each type.Indexes 0x0020–0x003F define fixed structure types (code DEFSTRUCT).Indexes 0x0040–0x005F define manufacturer specific structures (code DEFSTRUCT).Indexes 0x0060–0x007F (+ (D-1)*0x0040) define types (code DEFTYPE) for logical deviceD= 1..8.Indexes 0x0070–0x008F (+ (D-1)*0x0040) define structures (code DEFSTRUCT) for logical deviceD= 1..8.Indexes 0x0260–0x0FFF are reserved.Indexes 0x1000–0x11FF (0x1000–0x1029) belong to the standard communication profile, having specific meanings. (If logical devices are used, index 0x1000 links to indexes 0x67FF, 0x6FFF, 0x77FF, 0x7FFF, 0x87FF, 0x8FFF, 0x97FF, and 0x9FFF, for each logical device respectively.)Indexes 0x1200–0x127F define the SDO server parameters, including the CAN-IDs (data type specified by index 0x0022).Indexes 0x1280–0x12FF define the SDO client parameters, including the CAN-IDs (data type specified by index 0x0022).Indexes 0x1400–0x15FF define the PDO receive parameters, including the CAN-IDs.Indexes 0x1600–0x17FF define the PDO receive structures (or mapping), with sub-indexes specifying the components in key index:sub-index:numbits 32-bit unsigned integer format.Indexes 0x1800–0x19FF define the PDO transmit parameters, including the CAN-IDs.Indexes 0x1A00–0x1BFF define the PDO transmit structures (or mapping), with sub-indexes specifying the components in key index:sub-index:numbits 32-bit unsigned integer format.Indexes 0x1FA0–0x1FCF define object scanner lists for MPDOs, each being an array of up to 254 32-bit unsigned integers.Indexes 0x1FD0–0x1FFF define object dispatch lists for MPDOs, each being an array of up to 254 64-bit unsigned integers.Indexes 0x2000–0x5FFF define manufacturer-specific objects.Indexes 0x6000–0x67FF (+ (D-1)*0x0800) define standardized objects for logical deviceD(per device profiles, CiA 40x).Indexes 0xA000–0xAFFF define standardized network variable objects.Indexes 0xB000–0xBFFF define standardized system variable objects.Indexes 0xC000–0xFFFF are reserved.Thus, it seems to me it would make sense to implement the object dictionary facilities in distinct subsets, instead of a single big associative array.One for DEFTYPEs, one for DEFSTRUCTs, one for SDO parameters (indexes 0x1200–0x12FF), one for PDO parameters (0x1400–0x15FF, 0x1800–0x19FF), one for PDO structures (0x1600–0x17FF, 0x1A00–0x1BFF), and so on.One interesting approach for an associative array you might consider (especially for object dictionary keys 0x6000–0x67FF) is an array ofindex:type:descriptionstructures in RAM, sorted byindexin descending order, and populated at run time. This means you can use a binary search to look up any index very efficiently. Asindexis 16-bit anddescriptionis a pointer, that leaves 16 bits fortype, which should encode both the object-code (VAR, ARRAY, RECORD) and the C type of the structuredescriptionpoints to.VAR structures contain a single value, and ARRAY contains 1 to 255 values of the same type; in both cases the type specified by a single 16-bit unsigned integer (referring to a DEFTYPE object dictionary entry). For these, you could use a structure (in Flash or RAM) that specifies that type, the C/C++ type of the values, and a pointer to the value or array of values in RAM. The number of values would be encoded intype, so these would take up 256 of the 65536 possible values. If you also encode the C/C++ type in it (uint8_t,int8_t,uint32_t,int32_t,float), about 1280 of the 65536 possibletypevalues would be used.Optionally, an additional 256typevalues could specify the same, except instead of only the pointer to the first element, it would have a pointer to each variable; the variables all having the same C type.Optionally, an additional 256typevalues could specify the same, except instead of only the pointer to the first element, it would have a pointer to each variable, and a separate array of C type identifiers (maybeunsigned char) to define the type of each variable.For VISIBLE_STRING, and UNICODE_STRING VARs I'd use onetypes for immutable strings, and one for each possible mutable/run-time set string maximum length (255). For OCTET_STRING, I'd use 256types for immutable strings, and 256 for mutable strings, depending on the length. All together, these would take about 768typevalues.RECORD structures contain 1 to 254 values of different types. For these, you could encode the number of values intypealso, with the description consisting of an array ofpointer:ctype:deftypetuples, wherepointerpoints to the value in RAM,ctypeis its type, anddeftypeis the 16-bit DEFTYPE dictionary index for that sub-index. These would take 254 of the 65536 possible values oftype.Note that you do not need to define thousands of structures, as you can use variably modified types inside local scopes. That is,Code:unsigned char get1(int h, int w, int y, int x, unsigned char *m) { return m[y*w+x]; }unsigned char get2(int h, int w, int y, int x, unsigned char m) { return m; }unsigned char get3(int h, int w, int y, int x, void *m) {typedef unsigned char mtype;return (*((mtype *)m));}all typically compile to the exact same machine code.get2()uses a variably-modified type as a parameter, andget3()a variably-modified type in its body.Clang does not like variably-modified arrays in structures, so if you want to have two different arrays consecutively, useCode:// Better:unsigned char get1(int n, int i, void *data) {typedef int16_t atype;typedef int8_t btype;atype *const aptr = data;btype *const bptr = (void *)((char *)aptr + sizeof (atype));return (*aptr) * (*bptr);}// Not the GCC-specific:unsigned char get2(int n, int i, void *data) {struct {int16_t a;int8_t b;} *ab = data;return ab->a * ab->b;}

author: Simon
author_rank: Unknown
timestamp: February 25, 2024, 07:56:05 pm
content: i am completely confused and it feels like I end up with more memory and stuff to set up than I have. PDO's are dealt with already, I have a stucture for them and the message receiving code will just copy the data from the message to the correct jstructure location. What I really need to do with is sort out SDO communication so basically everything.Now the simple solution would be to just create an array as big as the number of possible indexes and sub indexes. This would mean that the index and sub index would simply address the array as the index, small problem 2^24 = 16 million and a bit, worse case each one is 4 bytes be that whatever, value or pointer it's 64MB, so not going to happen in 16kB of RAM or even the 128kB of ROM.OK so I would just map the sections I use but this would still be excessive and wasteful. What I need to do is remap the 24 bits that make up the index aand sub index of the objects I use but I cannot use the indexes or sub indexes themselves as locators.So the names of the objects I use are defines that translate to a unique number stating at 0. This number can be used as a unique identifier in any array. the goal is to type the object name in the program and that this works to make all the bits fall into place. as I have 6 types of data I can't create any structs if I want to use that data in my program. Alternatively when I search the array for an objects dictionary location definition by cycling through the indexes of an array that lists all of the index's and sub indexes I am interested in that array index number will be able to locate a pointer in a second array or the variable itself in the second array.The object dictionary definition is complete when the type of data is identified, CAN open already uses this sort of scheme for telling the node what objects each PDO should transmit.So:index : 2 bytess index : 1 bytevariable type: 1 byteA nice 32 bit variable. Or yea I could create an array of structs so that I can separately refer to index, sub index and data type without doing bit shifting. If i were using a lot of objects that use sub indexes (most just have an index with the sub at 0) this makes it easier to look through the indexes first then sub indexes. Worse case you have all 255 sub indexes used and now that is a long time spent searching.When it comes to the data itself I have two options. find a way of unifying it to one type or use pointers that in an array that the same index number will call upon. I currently do the latter. But here I use 4 bytes for the pointer, then there is the data. So if 4 bytes will hold every type (it will) then I may as well have that array of int32 and forget the pointers.So when I need to send a sequence of values for example to set up the slaves, instead of what I do now which is to write a line of code per item to send, I put all those objects consecutively in the array, this way I just step through the array incrementing the index until the last index.When data is received (SDO) I simply take the first 4 bytes and extract index, sub index and number of bytes, then I run through the array comparing the indexes, if an index matches I then compare the sub index, if successful I can compare the number of bytes received although this may be superfluous at this point so I look at my definition of the data type and copy the data into the correct data type before storing it in an int32 that is capable of being used directly in my program.For writing I simply send the index and sub index identified by the name that is a definition of the index in the array that contains the information along with the correct amount of bytes from the int32. total bytes used for any value sent 8.Now often I send the same object to more than one node, but often the value is the same for both sometimes different. When I receive data I need to have one variable per node. This does start to lead me into having an array of objects that are specific for say setting up and an array of the working objects that has an array of values so that unique values can be sent to nodes or received.Essentially here I am writing the CAN Open master that deals both with the communication DS301 and maybe others that are secret plus the DSP402 standard that relates to the motors I am using.

author: Nominal Animal
author_rank: Super Contributor
timestamp: February 26, 2024, 10:55:54 pm
content: Quote from: Simon on February 25, 2024, 07:56:05 pmI'm sorry if I confused you.Put simply, my point was that what CAN and CANopen call an "object dictionary" is actually a structured collection of entities that are interrelated in ways defined in the device profiles. So, in my opinion, it would make sense to create functions that implement such ways, as accessor functions (get, set), and associate those with the "object dictionary". For example:Code:// SPDX-License-Identifier: CC0-1.0 (i.e. Do Whatever You Want, just don't blame me)#include <stdint.h>// Basic types using CANopen numbering.#define TYPE_I8 0x02 /* int8_t */#define TYPE_I16 0x03 /* int16_t */#define TYPE_I32 0x04 /* int32_t */#define TYPE_I64 0x15 /* int64_t */#define TYPE_U8 0x05 /* uint8_t */#define TYPE_U16 0x06 /* uint16_t */#define TYPE_U32 0x07 /* uint32_t */#define TYPE_U64 0x1B /* uint64_t */#define TYPE_REAL32 0x08 /* float */#define TYPE_REAL64 0x11 /* double */#define TYPE_FLOAT TYPE_REAL32#define TYPE_DOUBLE TYPE_REAL64// Note: A different scheme would allow simpler conversion implementation// between numerically identified types, as when copying objects// to or from the object dict.// Accessor functions need some kind of *context*. It is a constant, but could have many forms.// This assumes __WORDSIZE == 32, i.e. sizeof (void *) == 4.typedef const struct {union {void *ptr;uint32_t u32;int32_t i32;uint16_t u16;int16_t i16;uint8_t u8;int8_t i8;};} ctx;// Object dictionary operations are stored in Flash, and describe accessor functions and their context.typedef const struct {ctx const context1; // This is usually a pointer.ctx const context2; // This usually uses the integer components. Say, a size of the array.int (*const get)(void *into, uint32_t index_intotype_subindex, ctx context1, ctx context2);int (*const set)(void *from, uint32_t index_fromtype_subindex, ctx context1, ctx context2);} object_dict_ops;// Each object dict entry takes (2+sizeof(void *)) bytes of RAM.#ifndef OBJECT_DICT_ENTRIES#define OBJECT_DICT_ENTRIES 500#endif// Recursive mutex/spinlock/interrupt locking.uint32_t object_dict_lock(void);void object_dict_unlock(uint32_t);// Object dictionary itself.const object_dict_ops *object_dict_entry; // In same order as object_dict_indexuint16_t object_dict_index; // In sorted order, largest index firstint32_t object_dict_entries = 0;const int32_t object_dict_entries_max = OBJECT_DICT_ENTRIES;// Each object dict index is handled by a set of operations.// These operations are added to the dictionary at run time, but defined at compile time.int object_dict_attach(const object_dict_ops *ops, int index);// Although only the index is needed for detaching operations,// it may be useful to keep the other parameters for verification.int object_dict_detach(const object_dict_ops *ops, int index);// Individual values can be copied to and from the object dict, per the accessor functions.int object_dict_get(void *into, uint_fast16_t index, uint8_t subindex, uint8_t into_type);int object_dict_set(void *from, uint_fast16_t index, uint8_t subindex, uint8_t from_type);Here is an example of accessor functions that can be used to access arrays ofint32_tvalues:Code:int get_i32(void *into, uint32_t index_intotype_subindex, ctx context1, ctx context2){const uint8_t into_type = (index_intotype_subindex >> 8) & 0xFF;const uint8_t subindex = index_intotype_subindex & 0xFF;int32_t *const array = context1.ptr;const uint8_t length = context2.u8;// Length requested?if (subindex == 0)return copy_i32_into(into, length, into_type);// Actual value?if (subindex <= length)return copy_i32_into(into, array[subindex-1], into_type);return -1;}int set_i32(void *from, uint32_t index_fromtype_subindex, ctx context1, ctx context2){const uint8_t from_type = (index_fromtype_subindex >> 8) & 0xFF;const uint8_t subindex = index_fromtype_subindex & 0xFF;int32_t *const array = context1.ptr;const uint8_t length = context2.u16;// Cannot set length.if (subindex == 0)return -1;// Setting an actual value?if (subindex <= length)return copy_into_i32(array + subindex, from, from_type);return -1;}They assume that functionint copy_i32_into(void *dst, int32_t src_val, uint8_t dst_type);copies the specifiedsrc_valinto*dstofdst_typetype; and that functionint copy_into_i32(int32_t *dst, void *src, uint8_t src_type);copies the value of*src_valof typesrc_typeinto anint32_tat*dst.They should also do any necessary type conversions.(Note that forcopy_into_i32(),dstis always aligned. Thevoid *can point to anything, so may be misaligned for the specified type.)In general, you'd need different accessor functions for the complex structures (like the one specified in index 0x0022), for device-profile STRUCTs (if any!), for ARRAYs (separate ones for each type you use in RAM), and for VARs (separate ones for each type you use in RAM).For example, to implement an array of 6 auxiliary variables for digital inputs at index 0x2050, you could doCode:int32_t aux_variables;const object_dict_ops aux_variables_ops = {.context1 = { .ptr = aux_variables },.context2 = { .u8 = { sizeof aux_variables / sizeof aux_variables, }},.get = get_i32,.set = set_i32,};At runtime, you would need to associate the ops with the desired object dict index:if (object_dict_attach(&aux_variables_ops, 0x2050)) { /* failed */ }Let's say you then have anint16_tv. You can useif (object_dict_get(&v, 0x2050, 0x01, TYPE_I16)) { /* failed */ }to get the first variable from 0x2050:0x01 tov, orif (object_dict_set(&v, 0x2050, 0x01, TYPE_I16)) { /* failed */ }to copy the value ofvto 0x2050:0x01.The same functions can be used for any number of arrays ofint32_t, each array only needs their ownconst object_dict_opsentry, to specify also the array location and size.Note thatobject_dict_attach()should keep the entries inobject_dict_index[]andobject_dict_entry[]––object_dict_entrydescribes indexobject_dict_index–– in decreasing order of index, so that all four functions can use a very fast binary search to find the target entry (or insertion index for _attach). All four functions also must internally uselockval = object_dict_lock();andobject_dict_unlock(lockval);to protect the accesses to the object dictionary (via a mutex, spinlock, or interrupt mask), so that it is safe to get and set the object dict values from any context.In your SDO receive and transmit functions, you'll useobject_dict_set()andobject_dict_get(), and if either returns nonzero/failure, you send the EMCY error message.Because the indexes are compact in memory (a single array ofuint16_t), a binary search will find them in less than 17 iterations (9 if you have less than 512 entries in use), and each iteration only involves an addition, a substraction, divide-by-two (one bit shift right), loading a 16-bit word from memory, and two comparisons. It will be fast. (Not as fast as directly referencing variables in memory, of course; but fast enough to use even in an interrupt context.)Quote from: Simon on February 25, 2024, 07:56:05 pmOkay, that simplifies things. Slaves must be modifiable by the master, but the master only needs to support different kinds of slaves. The above certainly applies, but you are unlikely to need the setters and only a single getter (as all your memory variables areint32_t's.)Something likeCode:// SPDX-License-Identifier: CC0-1.0#ifndef OBJECT_DICT_H#define OBJECT_DICT_H// If you modify the state associated with an object dict entry non-atomically,// you need to lock access to the object dictionary.uint32_t object_dict_lock(void);void object_dict_unlock(uint32_t);// Supported CANopen types#define TYPE_I8 0x02#define TYPE_I16 0x03#define TYPE_I24 0x10#define TYPE_I32 0x04#define TYPE_U8 0x05#define TYPE_U16 0x06#define TYPE_U24 0x16#define TYPE_U32 0x07typedef union {uint32_t key;struct {uint16_t index;uint8_t type; // CANopen type; entry is always int32_tuint8_t length;};} index_type;// Return the index:type:len corresponding to an index, or 0:0:0 if not found.index_type object_dict_find(uint_fast16_t index);// Copy a value from object dict into local storage, applying format conversions.// Returns the number of bytes copied, or negative for error.int object_dict_get_into(void *into, uint_fast16_t index, uint8_t subindex);// Copy a value from local storage to object dict, applying format conversions.// Returns the number of bytes copied from, or a negative error code.int object_dict_set_from(void *from, uint_fast16_t index, uint8_t subindex);// Associate an object dict index with a variable (len=0) or an array of variables (len=1..253).// Returns 0 if success, a negative error code otherwise.int object_dict_new(int32_t *mem, uint_fast16_t index, uint8_t length, uint8_t type);// Error codes; must be negative#define OBJECT_DICT_FULL -1#define OBJECT_DICT_INDEX_IN_USE -2#define OBJECT_DICT_INVALID_INDEX -3#define OBJECT_DICT_INVALID_SUBINDEX -4#define OBJECT_DICT_INVALID_TYPE -5#define OBJECT_DICT_INVALID_VALUE -6 // Value exceeds type limits#endif /* OBJECT_DICT_H */and the corresponding implementation (with a fixed maximum number of object dict entries):Code:// SPDX-License-Identifier: CC0-1.0#include <stdint.h>#include "object_dict.h"#ifndef OBJECT_DICT_ENTRIES#define OBJECT_DICT_ENTRIES 512#endifstatic index_type object_dict_index;static int32_t *object_dict_entry;int object_dict_entries = 0;const int object_dict_entries_max = OBJECT_DICT_ENTRIES;uint32_t object_dict_lock(void){// Recursive interrupt/spinlock.return 0;}void object_dict_unlock(uint32_t lock){// Recursive interrupt/spinlock.(void)lock;}// Internal binary search lookup. Object dict must be locked. Returns -1 if not found.static int index_at(uint_fast16_t index){if (index < 1 || index > 65535)return OBJECT_DICT_INVALID_INDEX;if (object_dict_entries < 1)return OBJECT_DICT_INVALID_INDEX;if (object_dict_index.index < index)return OBJECT_DICT_INVALID_INDEX;if (object_dict_index.index == index)return 0;if (object_dict_index[object_dict_entries - 1].index > index)return OBJECT_DICT_INVALID_INDEX;if (object_dict_index[object_dict_entries - 1].index == index)return object_dict_entries - 1;int i = 0;int imin = 1; // inclusiveint imax = object_dict_entries - 1; // exclusivewhile (imin < imax) {i = imin + (imax - imin) / 2;if (object_dict_index.index > index) {imin = i + 1;} elseif (object_dict_index.index < index) {imax = i;} else {return i;}}return OBJECT_DICT_INVALID_INDEX;}// Internal: Copy an int32_t to (unaligned) CANopen data buffer using the specified CANopen type.// Returns the number of bytes copied, or negative error code.static int copy_i32_into(void *into, const uint8_t type, const int32_t value){if ( (type == TYPE_U8 && (value < 0 || value > 255)) ||(type == TYPE_U16 && (value < 0 || value > 65535)) ||(type == TYPE_U24 && (value < 0 || value > 16777215)) ||(type == TYPE_U32 && value < 0) ||(type == TYPE_I8 && (value < -128 || value > 127)) ||(type == TYPE_I16 && (value < -32768 || value > 32767)) ||(type == TYPE_I24 && (value < -8388608 || value > 8388607)) )return OBJECT_DICT_INVALID_VALUE;unsigned char *dest = into;unsigned char data = { ((uint32_t)value),((uint32_t)value) >> 8,((uint32_t)value) >> 16,((uint32_t)value) >> 24 };if (type == TYPE_I32 || type == TYPE_U32) {dest = data;dest = data;dest = data;dest = data;return 4;}if (type == TYPE_I24 || type == TYPE_U24) {dest = data;dest = data;dest = data;return 3;}if (type == TYPE_I16 || type == TYPE_U16) {dest = data;dest = data;return 2;}if (type == TYPE_I8 || type == TYPE_U8) {dest = data;return 1;}return OBJECT_DICT_INVALID_TYPE;}// Internal: Copy a CANopen data buffer value of the specified CANopen type to an int32_t.// Returns the number of source bytes copied, or negative error code.static int copy_i32_from(void *from, const uint8_t type, int32_t *const to){const unsigned char *const src = from;uint32_t val;int len;switch (type) {case TYPE_I32:val = (uint32_t)(src)+ ((uint32_t)(src) << 8)+ ((uint32_t)(src) << 16)+ ((uint32_t)(src) << 24);len = 4;break;case TYPE_U32:val = (uint32_t)(src)+ ((uint32_t)(src) << 8)+ ((uint32_t)(src) << 16)+ ((uint32_t)(src) << 24);if (val & UINT32_C(0x80000000))return OBJECT_DICT_INVALID_VALUE;len = 4;break;case TYPE_I24:val = (uint32_t)(src)+ ((uint32_t)(src) << 8)+ ((uint32_t)(src) << 16);if (val & UINT32_C(0x00800000))val |= UINT32_C(0xFF000000);len = 3;break;case TYPE_U24:val = (uint32_t)(src)+ ((uint32_t)(src) << 8)+ ((uint32_t)(src) << 16);len = 3;break;case TYPE_I16:val = (uint32_t)(src)+ ((uint32_t)(src) << 8);if (val & UINT32_C(0x8000))val |= UINT32_C(0xFFFF0000);len = 2;break;case TYPE_U16:val = (uint32_t)(src)+ ((uint32_t)(src) << 8);len = 2;break;case TYPE_I8:val = src;if (val & UINT32_C(0x80))val |= UINT32_C(0xFFFFFF00);len = 1;break;case TYPE_U8:val = src;len = 1;break;default:return OBJECT_DICT_INVALID_TYPE;}// Compiler will optimize this to a simple assignment, but this way we're standards compliant.if (val & UINT32_C(0x80000000))*to = -(int32_t)(~val) - 1;else*to = val;return len;}// Return the index:type:len corresponding to an index, or 0:0:0 if not found.index_type object_dict_find(uint_fast16_t index){index_type result = { .index = 0, .type = 0, .length = 0 };if (index < 1 || index > 0xFFFF)return result;uint32_t lock = object_dict_lock();int i = index_at(index);if (i >= 0)result = object_dict_index;object_dict_unlock(lock);return result;}// Copy a value from object dict into local storage, applying format conversions.// Returns the number of bytes copied, or negative for error.int object_dict_get_into(void *into, uint_fast16_t index, uint8_t subindex){if (index < 1 || index > 0xFFFF)return OBJECT_DICT_INVALID_INDEX;uint32_t lock = object_dict_lock();int i = index_at(index);if (i < 0) {object_dict_unlock(lock);return OBJECT_DICT_INVALID_INDEX;}int32_t value;uint8_t type = object_dict_index.type;if (subindex == 0) {if (object_dict_index.length == 0) {// VAR access.value = object_dict_entry;} else {// ARRAY length is always UNSIGNED16 (UNSIGNED8 in older versions of CANopen).value = object_dict_index.length;type = TYPE_U16;}} elseif (object_dict_index.length >= subindex) {// Array value access.value = object_dict_entry[subindex - 1];} else {object_dict_unlock(lock);return OBJECT_DICT_INVALID_SUBINDEX;}object_dict_unlock(lock);return copy_i32_into(into, type, value);}// Copy a value from local storage to object dict, applying format conversions.// Returns the number of bytes copied from, or a negative error code.int object_dict_set_from(void *from, uint_fast16_t index, uint8_t subindex){if (index < 1 || index > 0xFFFF)return OBJECT_DICT_INVALID_INDEX;uint32_t lock = object_dict_lock();int i = index_at(index);if (i < 0) {object_dict_unlock(lock);return OBJECT_DICT_INVALID_INDEX;}int32_t value;int result = copy_i32_from(from, object_dict_index.type, &value);if (result <= 0) {object_dict_unlock(lock);return result;}if (subindex == 0) {if (object_dict_index.length == 0) {object_dict_entry = value;object_dict_unlock(lock);return result;} else {// Setting array length is not supported.object_dict_unlock(lock);return OBJECT_DICT_INVALID_SUBINDEX;}}if (subindex > object_dict_index.length) {object_dict_unlock(lock);return OBJECT_DICT_INVALID_SUBINDEX;}object_dict_entry[subindex-1] = value;object_dict_unlock(lock);return result;}int object_dict_new(int32_t *mem, uint_fast16_t index, uint8_t length, uint8_t type){if (index < 1 || index > 65535)return OBJECT_DICT_INVALID_INDEX;uint32_t lock = object_dict_lock();int i;if (object_dict_entries >= object_dict_entries_max) {object_dict_unlock(lock);return OBJECT_DICT_FULL;}if (object_dict_entries < 1) {i = 0;} elseif (object_dict_index.index == index) {object_dict_unlock(lock);return OBJECT_DICT_INDEX_IN_USE;} elseif (object_dict_index.index < index) {i = 0;} elseif (object_dict_index[object_dict_entries - 1].index == index) {object_dict_unlock(lock);return OBJECT_DICT_INDEX_IN_USE;} elseif (object_dict_index[object_dict_entries - 1].index > index) {i = object_dict_entries;} else {int imin = 1; // inclusiveint imax = object_dict_entries - 1; // exclusivewhile (imin < imax) {i = imin + (imax - imin) / 2;if (object_dict_index.index > index) {imin = i + 1;} elseif (object_dict_index.index < index) {imax = i;} else {object_dict_unlock(lock);return OBJECT_DICT_INDEX_IN_USE;}}if (object_dict_index.index == index) {object_dict_unlock(lock);return OBJECT_DICT_INDEX_IN_USE;} else {i = imin;}}if (i < object_dict_entries) {for (int k = object_dict_entries; k > i; k--)object_dict_index = object_dict_index[k-1];for (int k = object_dict_entries; k > i; k--)object_dict_entry = object_dict_entry[k-1];}object_dict_index = (index_type){ .index = index, .type = type, .length = length };object_dict_entry = mem;object_dict_entries++;object_dict_unlock(lock);return 0;}The idea here is that if you want to map an UNSIGNED16 VAR at index 0x6040 toint32_t status_word;, you doif (object_dict_new(&status_word, 0x6040, 0x00, TYPE_U16)) failed();Then, to copy the status word into two bytes at ptr, you doif (object_dict_get_into(ptr, 0x6040, 0x00) != 2) failed();Note that the return value, 2, is the number of bytes copied into ptr, and applies to all integer types. (I've only testedTYPE_I32, though.)Similarly, to copy the two bytes at ptr to the status word, you doif (object_dict_set_from(ptr, 0x6040, 0x00) != 2) failed();In the SDO receive/transmit functions, the return value will be negative if you reference an invalid index:subindex pair, and the number of bytes used from the buffer otherwise; this should make it rather easy to implement them.If you want to know the type at index 0x6040, useobject_dict_find(0x6040). It returns an union ofindex_type; the.indexis either0x6040or 0 if not found, and.typespecifies the CANopen type (TYPE_U16, above), and.lengthis 0 for VARs, positive for arrays.(That is,(index_type){ .index > 0, .type != 0, .length = 0 }refers to a VAR, and(index_type){ .index > 0, .type != 0, .length > 0 }refers to an ARRAY.Because the indexes are sorted in decreasing order, binary search is used, and time complexity for object access is O(N), whereNis the number of indexes registered in the object_dict. (The number of sub-indexes does not affect that at all.)I did test VARs and ARRAYs (length=2..253only) ofTYPE_I32, i.e.int32_tsome_data;withif (object_dict_new(some_data,index,length, TYPE_I32)) failed();and thatobject_dict_get_into(&len,index, 0x00)returns the length into an UNSIGNED16, and thatobject_dict_get_into(&val,index,subindex)andobject_dict_set_from(&val,index,subindex)copy the 4-byte values correctly.Theobject_dict_set_from()andobject_dict_get_into()always use little-endian byte order, as defined in CiA 301. The code should follow ISO C11 (actually, ISO C99 with the addition of unnamed structs/unions).If you have more than one thread, or the object_dict is accessed from interrupt context, then you should useuint32_t lock = object_dict_lock();andobject_dict_unlock(lock);around accesses to the variables added to the object dictionary, to ensure the accesses are atomic, except on architectures where (correctly aligned) accesses toint32_tare always atomic.

author: Simon
author_rank: Unknown
timestamp: March 31, 2024, 04:27:22 pm
content: Well I did work out a system in the end. I am heavily limited on RAM with just 16kB for the whole program. I created a struct type that has variables in it to store the index, sub index and a variable for the type that includes other markers too. The struct also has an int32 array of the value and an uint32 array to store the timestamps of each variable as it arrives.So the type variable has as it's least significant bits the number of bytes, bit 7 is the signed marker, any read only value has it's bit 6 set and I just added for more flexibility, setting bit 5 to alter the list of objects that get setup at the start so that I can choose whether to send the object to the slave node or skip it leaving the default.I did get left with having to setup a set of catch variables of each type so that if the value is say a signed 16 bit value it is copied from a i16 to the i32 rather than have the raw data copied from the can data as this would mean that the i32 value would become a random positive value if a negative i16 value is simply memcpy() in.At the start of the program I have to fill the array of objects with the data. I have defined the names of the objects to numbers starting at 0.It seems to be the best compromise. Decoding an SDO message can indeed take several hundred microseconds, usually around 270 µs, this is compared to a message transfer time of no more than 140 µs. The good news of course is that I can transfer most of the data I need with the PDO protocol.I suppose I could be more memory efficient by having separate arrays of different parts of the object dictionary. The main thing here would be simplification of the array structure but also that those elements that are set up at the start will probably be optimized into constant values in non volatile memory. So the indexes, sub indexes and variable types could each have their own array where the same object number will index to the data about the same object. They would likely end up in NVM and the arrays of values would go into RAM as well as the timestamp records.