Sending a valid (propperly arbitrated) CAN message to a car - Page 1
Forum section: electronics / general

author: Chicane
author_rank: Contributor
timestamp: November 18, 2024, 01:40:10 am
content: Hi, all! I'm working on a project of mine, involving reading the CAN messages in my car, but most-importantly sending messages emulating a press of a button on the dash. Determining the message to be sent is done, I have the hardware to send it with, too. My problem is, that the CAN messages to be sent and validated by the car require arbitration. The first 2 bytes of the message are some sort of counters, constantly changing its value. The second byte changes its value in a consecutive order, but the first byte appears as scrambled. Apparently it changes according to a formula, it's encrypted or some of the other modules on the Bus have some involvement in it. I can't figure this part by myself and need help with.Hoping someone with understanding and experience of the CAN protocol and communications in BMW G-series cars will read this post. Someone like Zucca maybe?I'm planning on using this on my personal car, so it has to be safe in the first place.I have logs and video of this CAN message that needs to be sent to the car. As I said, the problem is, that bytes 0 and 1 are constantly changing and those are the ones involved in the arbitration process, that needs to be figured out.Thanks in advance!

author: darkspr1te
author_rank: Frequent Contributor
timestamp: November 18, 2024, 07:15:17 am
content: if you google bmw g series canbus you get a lot of hits to side projects on the canbus.this site has esp32 code for just such a case.https://github.com/joeroback/racechrono-canbusdarkspr1te

author: JPortici
author_rank: Super Contributor
timestamp: November 18, 2024, 08:59:52 am
content: Quote from: Chicane on November 18, 2024, 01:40:10 amTrying to remove start/stop automatically or something like that? If we needed more reason to hate those giant touch screen panels.. in the past we would have just put a 555 on the wiring that emulated the press of the button.What we used to do is login to diagnostic functions and perform actuators test. Not possible anymore with G series as actuator control is now forbidden while the car is in motionAnd i think most of advanced diagnostic functions (data read/write) are only available in the ethernet channel, for example using ISTA, our CAN diagnosis are simply not being updated for BWM anymore.

author: neillnz
author_rank: Contributor
timestamp: November 18, 2024, 09:29:06 am
content: I have seen this same behaviour with a VW Golf. What I have seen is:The least significant nibble of the 2nd byte cycles from 0x0 to 0xF or 0 to 15. By the least significant nibble I mean bits 0 to 3.The 1st byte is some sort of constant that is tied to another byte(s) within the message and the 2 byte. Such that for every unique setting/payload there are 16 combinations of the first byte.For example the VW Golf message with an arb ID of 193 that is on the bus with the front light units:3rd byte is 00 when the turn signals are to be off3rd byte is 04 when the LHD turn signal is active and flashed off.3rd byte is 05 when the LHD turn signal is active and flashed on.3rd byte is 08 when the RHD turn signal is active and flashed off.3rd byte is 0A when the RHD turn signal is active and flashed on.3rd byte is 0C when the hazard/emergency warning lights are active and flashed off.3rd byte is 0F when the hazard/emergency warning lights are active and flashed on.For each of these 7 conditions there are 16 combination of the first byte, see the attached table.You might think that is bad but that is the set for when there is no front lighting on. There is another set of 7 for when the daytime running lights are on. There will be other sets but fortunatley for me I only need to bother with all off and DRL only so only 14 sets of 16. In my application I use a lookup table to retreive the first byte based on what the turn signals and DRLs are currently doing. Before I used the look-up table scheme I attempted to reverse engineer the first byte but I gave up.I think you will need to use a lookup method too.Is the message that you need to send one that only appears only when the button is pressed or does it occur on a regular basis? If it's an occuring one how do you intend to block it?

author: nctnico
author_rank: Super Contributor
timestamp: November 18, 2024, 09:49:47 am
content: No need to use lookup table. In CAN functions are typically indicated by using bits. So there is 1 bit that says flashing / turn signal is active, another bit to tell the lamp is on or off. Basic CAN messages can only hold 8 bytes at most so space is precious and no more bits than necessary are used.

author: neillnz
author_rank: Contributor
timestamp: November 18, 2024, 10:56:58 am
content: Quote from: nctnico on November 18, 2024, 09:49:47 amThat is correct for the byte(byte 3 in my exampe) which carries the information about the lamp function.The problem for the OP is that other bytes are constantly changing in sequence. The sequence of byte 2 is simple, just a count from 0 to 15(not including bits 4 to 7) but byte 1 appears to have no mathmatical relationship to the other bytes in the message. Or in other words byte 1 cannot be resolved by observing the other bytes within the message.To put it another way, when sending a message with the same information it is not always the same 8 bytes. Bytes 1 and 2 always change.The OP needs a method to predict what bytes 1 and 2 ought to be.From observing the 2024 VW Golf bus that includes the light units there are >100 messages that occur regulary, only a few of them use this sequencing method. In the 2023 Golf the same message did not use this complicated sequencing and there was not need for a lookup table.

author: Chicane
author_rank: Contributor
timestamp: November 18, 2024, 12:08:45 pm
content: Quote from: neillnz on November 18, 2024, 09:29:06 amThe entire message is dedicated to one single button, as far as I can tell. Byte 0 is scrambled, Byte 1 is the same as in your Golf. Byte 3 is confirmation of my button being acknowledged by the receiving module, the rest of the payload is FF. Pressing my button changes only one bit in the entire payload. Byte 0 is the one I'm scratching my head over. Doesn't change when other buttons/switches are triggered. Im almost convinced its a safety feature, a message validation by multiple modules. This is the Traction OFF button, which is likely being monitored by both ABS and BCM. I really want to disable this button, for the safety of my kid and his friends.

author: Chicane
author_rank: Contributor
timestamp: November 18, 2024, 12:20:37 pm
content: Quote from: darkspr1te on November 18, 2024, 07:15:17 amThis github rep is about reading OBD and PIDs. That's not a problem and there is no arbitration in the Bus at the OBD.Very different story at the PCAN Bus.I've seen plenty of people working on similar projects to theirs, but never found one explaining how to inject a message to to the internal CAN, without causing a major disturbance to the Bus. And I'm not trying to make the car stall or void its warranty. I just want to disable a switch.

author: Chicane
author_rank: Contributor
timestamp: November 18, 2024, 12:30:43 pm
content: Quote from: nctnico on November 18, 2024, 09:49:47 amThat's entirely correct. every single word of it.My problem is the handshake/validation/arbitration/encryption or whatever this byte 1 is. The car is ignoring my message, even if I transmit the last one i've read on that same Bus. And even if I'm able to figure out the next valid byte 1 to generate my message with, that will most-likely generate a fault when next message from the switch gets sent to the Bus, as it will be invalid.

author: Chicane
author_rank: Contributor
timestamp: November 18, 2024, 12:33:57 pm
content: Quote from: JPortici on November 18, 2024, 08:59:52 amNo, trying to disable the "traction off" button by sending Traction On to the CAN Bus repeatedly.

author: JPortici
author_rank: Super Contributor
timestamp: November 18, 2024, 12:36:33 pm
content: Quote from: Chicane on November 18, 2024, 12:33:57 pmSame principleQuote from: Chicane on November 18, 2024, 12:20:37 pmYou don't. You break the wire from peripheral and main bus and put your device in the middle. on the peripheral side it will read the frames from the peripheral to get the data you need to send back, and broadcast everything else from the main bus. On the main bus side it will imitate the peripheral (with the changes you are making of course) and it will acquire whatever else to broadcast to the peripheral side, as you don't know whatever the peripheral may need

author: Siwastaja
author_rank: Super Contributor
timestamp: November 18, 2024, 12:44:38 pm
content: Quote from: JPortici on November 18, 2024, 12:36:33 pmThis is probably the only way which would work here. Trying to "spam" an opposite command "in parallel" with the original peripheral is doomed to fail.I fail to see what is the whole purpose here, just instruct people not to press the button? I mean they can still drive the car of the cliff if they so wish. If they are irresponsible enough not to care about driving safely then surely they should not be given the car at all.

author: 2N3055
author_rank: Super Contributor
timestamp: November 18, 2024, 12:46:53 pm
content: If you want to disable "Traction control OFF" button, disconnect the button.Electrically.

author: Chicane
author_rank: Contributor
timestamp: November 18, 2024, 12:48:11 pm
content: Quote from: neillnz on November 18, 2024, 10:56:58 amMy problem is the handshake/validation/arbitration/encryption or whatever this byte 1 is. The car is ignoring my message, even if I transmit the last one I've read on that same Bus. And even if I'm able to predict the next valid byte 1 to generate my message with, that will most-likely generate a fault when next message from the switch gets sent to the Bus, as it will be invalid. Not sure what the consequences of that will be, it may cause a safety reset and stall the vehicle. Or if too many of those faults appear it may not recognize other important messages from the panel my switch is on. Not very safe to experiment with it, but I guess I'll have to.

author: Chicane
author_rank: Contributor
timestamp: November 18, 2024, 12:52:12 pm
content: Quote from: 2N3055 on November 18, 2024, 12:46:53 pmThis is not a Civic. I wish it was so simple. This button is on the same panel with the shifter and the emergency brake, all running on the same CAN.I also don't plan to glue the button, please don't suggest that either.

author: Chicane
author_rank: Contributor
timestamp: November 18, 2024, 12:57:40 pm
content: Quote from: JPortici on November 18, 2024, 12:36:33 pmI was considering that too, but then all the other vital functions from the transmitting module my switch is on, will be running through my filter. And that's not safe. In case of a failure, the car will have no shifter, emergency brake, camera and bunch of other controls. Otherwise it will be easy, yes. I only need to filter out 1 bit from 1 message.

author: SiliconWizard
author_rank: Super Contributor
timestamp: November 18, 2024, 07:45:40 pm
content: Quote from: nctnico on November 18, 2024, 09:49:47 amYes, CAN FD improves that significantly (up to 64 bytes payloads and 8Mbits/s), but one's not very likely to find CAN FD on anything but very recent cars; and not even all.

author: neillnz
author_rank: Contributor
timestamp: November 18, 2024, 10:24:46 pm
content: Quote from: Chicane on November 18, 2024, 12:48:11 pmYes, this is a problem. You cannot simply inject your message onto the bus while the original message is there. In the past I have had some success with sending my version of a message immediately after seeing the original(spamming as Siwastaja mentions), as a way of avoiding an arbitration conflict and to hopefully prevent the original message having an influence. For example a message is sent every 100ms to the headlight to tell it to keep the mainbeam on, upon reading the message I immediately send a message telling the headlight to keep the mainbeam off. This has worked for me in the past as a demonstration only, it's not great and far from a bullet proof solution as messages with a higher priority will take control of the bus extending the delay between the original message and my message which in my example could cause the headlight the briefly flash, plus other unforeseen issues.In your case you are dealing with a message that has another layer of call it integrity, in order to defeat this you are going to have to cut the bus and use a man-in-middle attack scheme. So you need hardware that acts a bridge between the two cut sides. You then manipulate the message on the way through. So this project is perhaps a lot harder than anticipated.You will need to cut the bus between the BCM that monitors the switch and the BCM that is looking at the switch condition, if this is the same BCM you are out of luck.As you say the solution you need comes with a serious safety risk. That risk far out weighs any gain to safety that disabling the switch would achieve, perhaps I can go as far as to misquote Dave here and say that the risk/benefit ratio is out "by orders of magnitude".They sure don't make this stuff easy for us!

author: Postal2
author_rank: Frequent Contributor
timestamp: November 18, 2024, 11:12:58 pm
content: Quote from: Chicane on November 18, 2024, 12:57:40 pmYou can't use ready-made CAN bus controllers, you need to physically disassemble the signals and insert your bits in the right places, like the acknowledgement bit.

author: Chicane
author_rank: Contributor
timestamp: November 19, 2024, 06:08:09 am
content: Quote from: Postal2 on November 18, 2024, 11:12:58 pmI'm no CAN expert, but from my experience and other experiments I've previously done, I have to say you're wrong. Maybe it's a language barrier, since you're talking about signals and CAN controller. You can type in Russian, I can read and understand, but can't write in it.All I need is one MCU and 2 CAN transceivers in terms of hardware. I have a few of those left over from other projects. Then from the CAN message spit out by the switch, I can just filter out 1 single bit, that changes when the switch is depressed. -> 0x123 AA BB CC 01 FF FF FF FF ->  -> 0x123 AA BB CC 00 FF FF FF FF -> That way the acknowledgement bit (AA) is untouched, but the switch position bit (01) is replaced with 00 by the filter, as if the switch was never touched.This is very easy to do, much easier than reading an acknowledgement byte, calculating the following one (if lucky enough to obtain the formula or encryption key), then reading the second byte and generating the next valid byte, hoping the rest of the modules will be kind enough to validate it.My problem with the option involving a filter is, that if it fails or bugs out, the car may stall as the worst case scenario. I'm loosing control over the shifter, emergency brake and Park button in the best case scenario.

author: Chicane
author_rank: Contributor
timestamp: November 19, 2024, 06:30:16 am
content: Quote from: neillnz on November 18, 2024, 10:24:46 pmI hate to say it, but you're right, unfortunately. It will either be an unsafe filter solution or an extremely complicated arbitration challenge, involving a ton of research and testing. This makes it a stupid idea, but that was clear to me from the very beginning. It's not that much about it being worth my time and effort, just to disable the fkn switch. What makes it worth is, that I will hopefully figure out the arbitration part. Not only that will help me with future projects I have, but will give me an unmatched satisfaction I get by solving this kind of problem. I like a challenge and that's a fairly big one for me, I'm ready to pay the price for it.

author: Chicane
author_rank: Contributor
timestamp: November 19, 2024, 07:18:22 am
content: Quote from: Siwastaja on November 18, 2024, 12:44:38 pmI don't know how familiar you are with the CAN protocols and the specific structure of the proprietary messages in this generation BMW, but I't unlikely you can predict the outcome of spamming the Bus with a message opposing the one sent by the switch. This is a car. It needs to be safe and reliable, resilient to failures of individual modules, loss of data packets on an of the data Busses and what not. If you disconnect a node from the Bus and then reconnect it while the car is running, it may show an error on the IC but this node will be recognized and continue to function as it should. There are countless arbitration bytes that will have to be reset in all nodes on that Bus, in order for this node to operate again. Therefore there is hope, in my opinion, but will take quite a bit of experimenting to find out how to reset it.There is a chance I don't need to reset it, it will reset with the next sleep cycle, or even auto-rest with the next valid message from the switch.All I'm trying to say is, that there are so many variables and conditions, you have no way to predict if and what will work. I was only hoping to find someone familiar with the proprietary arbitration process, the rest is on me to test out.

author: elektryk
author_rank: Regular Contributor
timestamp: November 19, 2024, 08:32:09 am
content: It reminds me a bit this topichttps://www.eevblog.com/forum/projects/throttle-attenuator/

author: Chicane
author_rank: Contributor
timestamp: November 19, 2024, 08:40:58 am
content: Quote from: elektryk on November 19, 2024, 08:32:09 amLOL! I've done that before and it was working perfectly.It's sad to see again how many people comment with no clue on the subject.

author: Postal2
author_rank: Frequent Contributor
timestamp: November 19, 2024, 09:09:54 am
content: Quote from: Chicane on November 19, 2024, 06:08:09 amI agree, we can try. Aliexpress sells filter boards that give false mileage, as well as developer boards, I bought several of these:https://aliexpress.com/item/1005001620616382.htmlBut if the messages transmit time stamps, then this will not work. Then you can adjust as I wrote, it will work for sure, although non-trivial and difficult.The difficulty of the circuit I mentioned is that it requires keys that break the CAN bus, since we cannot adjust the bits when the transmitting device is listening to its transmission. Although again it is necessary to check, perhaps the program does not check this controller flag. Then it is not necessary to break, but keys are needed to pull the bus to the recessive state.

author: JPortici
author_rank: Super Contributor
timestamp: November 19, 2024, 10:10:56 am
content: Quote from: Siwastaja on November 18, 2024, 12:44:38 pmUsed to be able to switch traction control off permanently, used to have the physical damn button instead of menu diving (there are situations in which you DO NOT want traction control or you can't get out of snow/mud patches) or disable start/Stop as in some models it wears out the starter, or you don't want the car to die at a roundabout, things like thatso you want ways to disable the "features" automaticallywe pondered about making a product for that but in the end we didn't bother, much less hassle to get the thing already made from aliexpress, as it has a very low margin.

author: JPortici
author_rank: Super Contributor
timestamp: November 19, 2024, 10:16:36 am
content: Quote from: Chicane on November 19, 2024, 07:18:22 amOh, so you still have buttons? then, operate on the buttons and leave the can bus alone. Put it simply youcan'tstop a module from sending a frame by messing up arbitration. Even if you did that (and you'd need an FPGA to monitor the bus and mangle the right bits when appropriate, then wait interframe space then send your frame) after a few times the node you're stopping will flood the bus with error frames because it can't send, or because your mangled frame is wrongly formed or has wrong data. don't do that. Put yourself in the middle, replicate and change the data, this is how it's done.Or act at the source and disconnect/change/whatever the physical buttons.

author: tom66
author_rank: Super Contributor
timestamp: November 19, 2024, 11:54:33 am
content: Put a piece of tamper-evident tape over the button.If your son (?) removes the tape to activate/deactivate the function he loses driving privileges until you are convinced he is safe again.Alternatively, buy a really cheap 1L petrol car with a 5 speed manual and give that to him instead of a sporty BMW.You are trying to solve a problem with technology when it is solved by parenting - Ferris Bueller shows it isn't possible!

author: Chicane
author_rank: Contributor
timestamp: November 19, 2024, 06:38:57 pm
content: Quote from: JPortici on November 19, 2024, 10:16:36 amI'm afraid you misunderstood what I'm trying to achieve here. It's not about preventing the switch (sender) to deliver its messages to the can by flooding the Bus with scrambled messages in order to jam it. That would be the most-stupid way of doing it.I was thinking to send message "switch off" immediately after the switch itself sends a "switch on" message to the Bus. And my message has to be properly arbitrated and recognized by all other nodes on this Bus. The arbitration is the hard part to do. I don't want to disturb the Bus and trigger errors, if possible. My point was, that even if I manage to send my messages properly arbitrated and not cause errors, that will throw off the arbitration of the messages sent by the switch itself, and still cause errors. I don't know what effect those errors will have on the BCM and the module this switch is on.

author: Chicane
author_rank: Contributor
timestamp: November 19, 2024, 06:55:51 pm
content: Quote from: Postal2 on November 19, 2024, 09:09:54 amI have those "filters" from Aliexpress and can easily modify their code to replace the "switch On" byte with a "swith off" byte in the message. That's why I said that's very easy to do. There are no keys requires when doing that, as the message on the input of the filter is the same on the output, except this one byte. The arbitration of the switch messages will remain intact and no errors or flags.You're talking about controller flag and recessive mode, which i don't understand why it is involved, if the filter is a simple repeater of the existing messages in and out of the switch. The time stamps will also remain intact, if the message contains them. And then keys to break the Bus? What is that about? I'm confused. Did you mean a key to brake the message and modify it? That doesn't make sense to me.Again, I don't consider myself an expert. It's possible you know a lot more than I do and I just don't understand this part. But at the moment i have a feeling we are talking about different things. Different approaches to achieve the goal.

author: Postal2
author_rank: Frequent Contributor
timestamp: November 19, 2024, 08:05:09 pm
content: Quote from: Chicane on November 19, 2024, 06:55:51 pmThe solution to the question in the form in which you ask is 10 times more complicated than even my complicated solution. But the question itself is interesting.

author: neillnz
author_rank: Contributor
timestamp: November 19, 2024, 10:51:05 pm
content: Quote from: Chicane on November 19, 2024, 06:38:57 pmAre you still planning on implementing the scheme where you place your message onto the bus without physically breaking it? This cannot work for you because the particular message you are working with uses another layer or integrity(byte 0 sequence) than will not support this method. You must physically disconnect the the module from the bus and use a filter aka man-in-the-middle-attack scheme. I have sent you a PM.

author: nctnico
author_rank: Super Contributor
timestamp: November 19, 2024, 10:59:42 pm
content: I don't think that is entirely true. The receiver can't know the sequence the sender is at and messages might be missing. So maybe overruling each message as if there are two units on the bus might work after all. Another option is to send a traction control on message to the console with the buttons which then reverts back to sending 'traction control on' message. When the car is started, some initialisation needs to be done and I suspect the default is to have traction control on. IMHO far more research is needed to have a definitive answer on what will work and what not. Keep in mind that car manufacturers like things to keep working even if half the CAN bus is messed up.

author: Postal2
author_rank: Frequent Contributor
timestamp: November 20, 2024, 12:37:48 am
content: I still assume that the filter on the microcontroller with two CAN interfaces has too much delay. Because first the message will be fully received in the FIFO, then copied to the outgoing FIFO on the second line, and then the transmission will begin. On the FPGA, you can perform redirection immediately. Even such a delay is easy to see, but it is unlikely that the bus arbiter will be configured to detect it. Considering that the scheme is much simpler than substituting bits on the fly, most likely the filter on the FPGA is the preferred option.In addition, it will be easier to simulate arbitration on FPGA when signals appear on two interfaces simultaneously, since when transmission starts on one interface, the second interface appears free.Quote from: nctnico on November 19, 2024, 10:59:42 pmThe controlled block will send back the status of the condition, which also needs to be filtered.

author: neillnz
author_rank: Contributor
timestamp: November 20, 2024, 03:07:14 am
content: Quote from: nctnico on November 19, 2024, 10:59:42 pmThis is a special case. In this case the receiver does know the sequence. The bits 0 to 3 of byte 1 count from 0 to 15. Byte 0 is a unique byte that changes each time the message is is sent and depends on the payload including byte 0. So if the data part of the message is constant(i.e. switch constantly off) you will observe 16 different messages in a repeating sequence. What you will find is that almost all the other messages on the bus do not work like this and as you say the receiver cannot know the sequence of those messages/frames. I have some (perhaps too much) experience with this. Not all cars have this convoluted scheme. When I first came across it it sure did create some headache!

author: Postal2
author_rank: Frequent Contributor
timestamp: November 20, 2024, 05:22:23 am
content: Quote from: neillnz on November 20, 2024, 03:07:14 amI have experience wasting time with this. Here is a simple switch in the photo. And I thought, what problems could there be to switch on the sockets I need via CAN-bus. This was necessary for testing. As a result, I could not do anything. To check, I uploaded my firmware and switched using my own code.

author: nctnico
author_rank: Super Contributor
timestamp: November 20, 2024, 04:54:56 pm
content: Quote from: neillnz on November 20, 2024, 03:07:14 amYes, but what I mean is that the system still needs to work if -say- half of the messages is missing. This means that whatever sequence there is in the 'magic' byte, the receiver needs to be somewhat leanient in as it won't always receive messages in the right order. It is also possible messages are received more than once if the transmitter is configured to repeat messages which where not acknowledged for some reason.