Reading data from mouse/keyboard + emulating a mouse/keyboard - Page 1
Forum section: electronics / general

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 12, 2016, 09:16:33 am
content: How hard would it be to make a device where I can plug a mouse/keyboard into it, it reads it's data, then emulates a mouse/keyboard and plugs into a computer and sends that data? What do I need to do, and what is some good reading material?Essentially what I want to make is a device where I plug my mouse and keyboard into it, and it has 3 sets of outputs that go to 3 different computers. Those computers see it as their own set of mouse/keyboard (ex: it's always connected) but on the device I have 3 buttons, and depending on which one I press, the real mouse/keyboard will send the data to only the one PC. Essentially kinda like a KVM, except I don't want it to disconnect/reconnect each time, I just want the PCs to always see a mouse/keyboard connected, it's just that the device itself will only send the data to one of them, the others will see it as if they mouse/keyboard were idle. There would be 3 monitors in this setup, each connected to their respective PCs, so I'm only looking at controlling the mouse/keyboard but it needs to be seamless so the instant I press the button it's ready on the computer. The buttons may be replaced by some other process such as software that can detect when the cursor reaches the edge of one of the screens and sends a signal to the device, so it needs to be fast. That's something I would figure out separately, either communicate via the same USB link, or a separate one. I'd probably stick with the buttons for the time being.I'm not too sure where to start so I'd like to be pointed in the right direction. I imagine I need to read up on Human Interface Device standard as a start. I already have some Atmega128 chips and the programmer, so I'd like to stick with atmel, if this can be done fully at the MCU level, but I presume I may need/want other discrete chips as well.I'm fairly new at electronics so this will serve as a learning project while solving a problem I'm having at the same time.I eventually want to learn how to do that at the video level and make a KVM (for a different purpose), but I imagine that would be getting into FPGAs, I'm not quite ready to dive into those just yet.

author: westfw
author_rank: Super Contributor
timestamp: October 12, 2016, 10:22:35 am
content: Ps/2 or USB? PS/2 would probably be pretty easy, but uCs with more than one USB port are pretty rare, and i don't think I've ever seen 4.Maybe Ps/2 on the micro, and cheap ps2/USB adapters to connect to the actual pcs?

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 12, 2016, 10:53:51 am
content: I want to go USB, so it does not restrict my choice of hardware. Motherboards with two PS2 ports are pretty rare now and mice/keyboard with said ports are too. Don't want to deal with adaptors, that just adds extra complexity and possibility of having problems. (ex: certain mice/keyboard / OS / etc may not work well with them)I'm not opposed to using one MCU per computer, they are fairly cheap. I can then use i2c or something to communicate between them.

author: technix
author_rank: Super Contributor
timestamp: October 12, 2016, 10:57:14 am
content: I would suggest you using more than one microcotrollers and network them, so you can use cheaper components and get an easily extensible system. To each downstream port you have a smaller microcontroller that converts commands from the internal bus (e.g. I2C) to USB. You can even use the ready-made Arduino code for ATmega32U4 or ATmega16U2 for this purpose. Then you need an upstream device that have USB OTG feature like STM32F105 and use that to interface the keyboard and mouse (Don't worry about being limited to one port, since you can use USB Hubs to increase port count almost for free.) And since mouse and keyboard don't have a lot of data bandwidth requirements, as long as you don't plan to play games using that, you can use the normal 100kbps I2C bus as the internal bus.

author: rob77
author_rank: Super Contributor
timestamp: October 12, 2016, 11:07:51 am
content: what's wrong with Synergy ? it's a multi-platform software solution - i'm using it because i often work on 2 computers simultaneously (sometimes even 3).. in addition it has some nice features - like sharing clipboard. and costs only $10 one time fee (29$ with support and SSL encryption).http://symless.com/synergy/the only thing i need to make myself is s ctrl+alt+del "keyboard" which will be connected to a windows laptop i'm using because the corporate policy settings doesn't allow me to enable accepting "software" ctrl+alt+del... it has to come from a HID device physically connected to that laptop. but if you don't have windows with enforced policy from a domain , then even ctrl+alt+del works on a login screen with synergy.btw.. planning to make the single-button ctrl-alt-del keyboard with a ATSAM3S - once i will have the time for it

author: Jeroen3
author_rank: Super Contributor
timestamp: October 12, 2016, 11:13:42 am
content: Isn't this precisely what a luxury KVM does?

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 12, 2016, 08:32:06 pm
content: Quote from: rob77 on October 12, 2016, 11:07:51 amThe version I have crashes all the time. That's actually what I was using. The newer version has too many dependencies and I can't get it to install. So decided to basically look at a hardware solution. Nice thing with that is getting "physical" level access too in case I need to get in the bios or something I can. I could technically have one PC that is installing an OS for example, and still control it.

author: Rasz
author_rank: Super Contributor
timestamp: October 12, 2016, 09:28:56 pm
content: im pretty sure beefier KVMs do this already (no disconnection, essentially proxy)

author: C
author_rank: Super Contributor
timestamp: October 12, 2016, 10:46:46 pm
content: I would suggest using a micro controller with USB & CAN interfaces.With I2C or SPI you ending up with polling other controllers.With USB OTG you can be a host or device on usb. Nice ones let you shift modes based on what is connected.CAN is an event buss, just what you need here.A USB Keyboard is not a simple device. You think of the keys but you also have the lights(caps lock, num lock, ...). Each key creates two events(pressed, released) and keyboard mode can change the data sent with those events.Using CAN buss your system just got very easy. A USB event becomes a CAN event and back to USB event. Your buttons become CAN event which tells who on CAN buss should pass ____ CAN event on to connected USB.All micro controllers could be the same and in one or more boxes. You can add more. You could use one for keyboard and one for mouse or a hub to do both.You also have many options for your buttons. Cheap buttion option connected to micro controller, a separate button micro controller to CAN, a button micro controller to USB or a keyboard. With the buttons option added to controller connected to PC you have a choice of many keyboards.Very expandable.EDITForgot the USB Keyboard to Serial, USB to serial and CAN to serial.

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 13, 2016, 02:42:40 am
content: I'd have to read up more on CAN. But yeah I would probably design this to be fairly modular so I can just plug as many together as I want.My thinking is that I don't really need to encode the USB packets much, I just need a way to present a USB device to each machine, then sit and wait, and connect to a real USB device. Well I'd probably want to make it two way as I think keyboards are two way. (ex: the LEDs for caplock etc)What I'm thinking it could do is when you plug a device in it, it would gather all the basic details like vendor ID etc, then it would present that device to all 3 computers just to let it know it's there, so it loads the appropriate drivers etc. But from then it would only communicate with the one in which it's button is pressed.Is there purpose built USB chips that could help in making something like this? I do have atmega128 and MCP 2221 chips already. I'm thinking the MCP 2221 could maybe present the HID device to each computer. (one chip per computer) not sure about what chip would read data from the mouse/keyboard though.Idealy I might want to look at a chip that can act as a USB hub too, so I can have the mouse and keyboard on one device and each computer on one cable. But perhaps to keep it simple I'll keep it separate for the keyboard/mouse end. Lot of possibilities there. For now I will start as a simple device where if I plug it in a computer it sees a HID device, and if I plug a mouse or keyboard in the device it just passes on the info to/from the computer. Once I can get something like that working it's just the thing of expanding it. Just not too sure where I'd start, what I should read up on, etc.

author: C
author_rank: Super Contributor
timestamp: October 13, 2016, 06:12:08 am
content: Have not looked at MCP 2221 much.States it is a composite device and what operating systems support it. This may might could prevent use during boot. To add more computers is large change.USB controller built into micro controller gives you options a canned thing like MCP 2221 can not do. Also means more software needed(USB controller code).Think with AVR you have a choice of USB or CAN not both. USB I think would be best choice here as you can find examples of being a keyboard or a mouse. Then you have to find a CAN CONTROLLER that you can attach and create software or has examples you can use.Think cheap is micro controller with both built in and has examples on how to use. You can find a lot of ARM micro controllers with both and examples at low cost.ARM, USB-OTG & CANYou should note that a mouse and keyboard are two separate usb devices. Putting them together you have a Hub with two devices or a USB Composite device. Ether could be a fail at boot start while working at some point.Note that I do not remember a USB device controller acting like a hub with two devices.Stating the obvious, need to keep it simple and have an out if one option fails. Would be good to prove idea works before doing special boards. From what I have seen, many of same pcboard can be cheaper if you decide to build.So to your post.USB stops at your USB host micro controller.USB starts at your USB device micro controller.There are no USB Packets just data to pass between the two above.CAN is an eventID and up to 8 bytes of data.When your host receives keyboard vendor IDYou send a CAN packeteventID = keyboard1_vendorID_tag a fixed integerdata= vendor IDthis simple for each thing the host receivesThis should beeventID = device1_vendorID_tag a fixed integerdata= vendor IDNow not specific for a keyboard or mouseHere CAN is great for you. A new USB data item is just a new CANID. It is sent to all devices connected to CAN Buss, so it's each device deciding if to use it on USB now, store it for later use on USB or send to trash.The LED's and other data from PC becomesSave a local copy.Send on CAN if currently selected.Send on CAN IF changed to selected.For cheap get a cheap micro controller board thatCan Be a USB host (OTG) and has accessible connections to CAN connections.This is enough to test basics with.More cash, get two boards and two CAN Driver chips.This is enough to test Keyboard link to PC or Mouse to PC.With three boards, you have Keyboard & Mouse to PC.With a USB hub you can test if One cable to PC will work.Having examples ofUSB-OTGCANUSB HUD deviceUSB keyboard deviceUSB mouse deviceWill give you a better chance to it get working. Can also save you a huge amount of time.Might look at STM32 discover boards to start with that have USB-OTG & CAN. A lot of different ones for cheap.Hint you should see two USB connectors not one and then need to check for OTG and CAN. One usb connector is for the STlink programmer connection. If it has a Mems sensor it could have a tilting mouse example.

author: Jeroen3
author_rank: Super Contributor
timestamp: October 13, 2016, 07:07:04 am
content: STM32F1's don't do CANandUSB. You must get F4+.Instead, get an LPC1768, does CAN, and has two USB's. With the ease of mbed's library to set it up in an hour.

author: westfw
author_rank: Super Contributor
timestamp: October 13, 2016, 08:52:00 am
content: Quoteuseless for USB :-(QuoteYou will be, in essence, implementing adaptors on your own, then. One micro with USB Host to talk to your keyboard, N micros with USB Device to talk to your PCs, and some sort of communications protocol between them.

author: technix
author_rank: Super Contributor
timestamp: October 13, 2016, 10:06:55 am
content: By the way, you can have a look at ATEN CS1794 4-port USB-HDMI KVMP switch. I have used this KVM switch for a few years already and it is very reliable for me. It switches USB peripherals and HDMI A/V signals, and I do have 4 computers hooked on it at the same time.

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 13, 2016, 10:41:44 am
content: Quote from: westfw on October 13, 2016, 08:52:00 amI meant I don't want to be messing with ps2 and usb adaptors, I just want to do everything USB. Of course there might be some "adapting" going on at the MCU level but that's not nececerily what I meant.Quote from: technix on October 13, 2016, 10:06:55 amProblem with KVMs is that they switch too slow and if switching a lot or fast back and forth they lock up. I have one for the main monitor to switch between my Linux and windows box, and for example if I switch to Windows and then back to Linux, I need to wait about 10 seconds before I can switch to windows again, or I lose mouse/keyboard completely. Basicallty they just plug/unplug the mouse/keyboard/monitor from the computers so it's equivalent to manually hot plugging the devices. KVMs tend to be really flaky like that. I need this to be seamless, as the goal is to eventually automate it through software so when cursor hits the edge of the screen it automatically switches to controlling the other PC. The monitors will be physically connected to each machine so I'm only looking at switching keyboard and mouse. Whatever solution I end up going with, I will probably start by keeping them separate just so I don't have to worry about USB hubs etc... but the final design would probably have a hub for each PC, actually I can put the hub first, then switch the USB itself, different ways I can approach it. I checked digikey real quick and they do have dedicated chips for USB hubs so I should be able to use that I imagine.Though this is sounding like it might be more complex than I was hoping, so I might try a full software solution again even if I end up having to write my own. Basically the end goal is to have 3 displays that are independent of each other. ex: I don't want windows/dialogs that are launched from one screen end up on another. Tried all sorts of things like separate X session and different window managers and they all violate this simple concept, so only 100% solution seems to be to have 1 actual PC per monitor. The Raspberry PIs will probably be acting as some kind of thin client.Though still open to various hardware based options and it would make a neat project. I'm on Linux as well so whatever solution I go with needs to be Linux friendly as far as development goes. I tend to stick to AVR for that reason.

author: eugenenine
author_rank: Frequent Contributor
timestamp: October 13, 2016, 11:11:56 am
content: Quote from: Red Squirrel on October 13, 2016, 10:41:44 amRun each OS inside Virtualbox. You have one Linux host and it brings up a virtual guest for each linux and each windows guest you need and runs each inside one monitor.

author: technix
author_rank: Super Contributor
timestamp: October 13, 2016, 12:22:41 pm
content: Quote from: Red Squirrel on October 13, 2016, 10:41:44 amMy CS1794 responds within 2 seconds to any button press and have internal rate limiting. It uses electronic switching with internal buffers too not unlike your design (those that lock up uses mechanical switching or lacked internal buffers)For me my A/V receiver (sitting in between my display and the KVM switch, stripping the audio out from the HDMI stream) cannot usually keep up with this fast-action KVM switch and takes up to 5 seconds to catch up. If I bypass the A/V receiver and plug my display (an 8-year-old Lenovo OEM one which does not have HDMI. I have to use a decoder box to strip HDCP off the stream before sending it into the DVI port on my display) directly into the KVM switch it responds within 2 seconds too.

author: Rasz
author_rank: Super Contributor
timestamp: October 13, 2016, 08:03:25 pm
content: Quote from: Red Squirrel on October 13, 2016, 10:41:44 amagain - your problem is with shit hardware you are using, not KVMs in general

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 14, 2016, 06:00:57 am
content: Well when it comes to USB/DVI KVMs, the choice is quite limited so they pretty much all are shit. When you search for KVMs 90% of results use PS/2 and VGA. Ridiculous actually. Most motherboards don't even have PS2 anymore and most decent monitors/video cards don't have VGA anymore.If I can manage to do this project, I'd probably look at making a KVM too as a separate project as I can use one of those too but my existing one sorta gets the job done. Though the video switching part may be a bit more challenging if I want to emulate and not just physically switch like KVMs do. That's getting into very high bandwidth stuff.

author: C
author_rank: Super Contributor
timestamp: October 14, 2016, 12:10:15 pm
content: Might want to keep in mind that the USB host & device you are creating here does not have to understand keyboard or mouse attached.Think of a keyboard.The PC sends some packets to keyboard. Keyboard responds with packetsYour board does this.When you connect micro controller USB Device to PC, The PC sends some packets to micro controller USB Device(keyboard). micro controller USB Device copies to micro controller USB Host and sends. Keyboard responds with packetsThis micro controller relay just goes both ways.Some types of packet data gets stored to remove need to Connect on PC Change.Need to keep everything simple.A keyboard connect causes a reconnect to PC so that PC is working with correct keyboard data.No where above is this specific to a Keyboard, so a mouse or other device should function. Some device(s) should not be used in this way.When this test setup is working, you change it to copy from USB Device to/from CAN & USB Host to/from CAN and use two boards.Note think it will be easer if you start with a micro controller that has some type of display so you can get feedback on which half of above is working. Think you will find that STM32 discover boards that have a display also have two usable USB connections & CAN.Should get cheaper if micro controller used has two USB-OTG & CAN.Even Keeping it this simple, you should expect errors that will need work arounds. Time between two USB packets is one area.

author: Rasz
author_rank: Super Contributor
timestamp: October 14, 2016, 09:25:31 pm
content: Quote from: Red Squirrel on October 14, 2016, 06:00:57 amno, just yours, its clearly broken (hanging) and shitty (10 second switch time)Quote from: Red Squirrel on October 14, 2016, 06:00:57 amehh... if you really really want it. go ahead, here are ready made off the shelf building blocks:https://github.com/cnlohr/espusb$2 module that does USB mouse and keyboard emulation + wifi link. you will need as many as you have computers to connectplus an SBC with wifi and usb. for example eitherhttps://www.adafruit.com/products/3055orhttps://www.adafruit.com/product/2885+ usb hub + usb wifiand about 100 lines of python to facilitate switching and coordinate endpoints. No assembler, no C, no learning about usb stack, just putting modules together and making something work over two weekends.Quote from: Red Squirrel on October 14, 2016, 06:00:57 amemulate what? you dont need to emulate vga (resistor and fixed resolution is enough to make sure vga is active), hdmi switcihng can be as fast as 1s if you dont need(or strip) hdcp

author: C
author_rank: Super Contributor
timestamp: October 14, 2016, 09:43:09 pm
content: QuoteA very nice security hole for hacking.

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 15, 2016, 06:08:29 am
content: Quote from: C on October 14, 2016, 12:10:15 pmYeah that is kind of what I have in mind, basically the MCU just acts as a relay of USB data. For displaying info the easiest route would probably be a separate serial/uart USB setup and I'd just have a console up. Could also look at existing products and what chips they use and go from there. A crude way would be to look at a various mouse/keyboard related products to see what kind of chips they use and go from there. Basically if I can control a mouse/keyboard through another one, I have a basic foundation. But I can probably build it to not care about the type of device just that it is a USB HID device.The neat thing after figuring out the basics would be to make it modular and configurable. There would be optional software on the machines that can tell the device when cursor is at a screen edge. Essentially this would be a hardware level Synergy. Idealy I could even have it work at the Bios level. I can't see why not if it's just relaying USB communication. Of course I'd want the hardware buttons on the device at this point.

author: Rasz
author_rank: Super Contributor
timestamp: October 15, 2016, 10:44:38 pm
content: Quote from: Red Squirrel on October 15, 2016, 06:08:29 amyou cant do that, you can emulate endpoints, cache responses, but not relayto understand why you would need to learn low level usb (1-2 months maybe?)Quote from: Red Squirrel on October 15, 2016, 06:08:29 amalready linked you off the shelf $2 module that does thishttps://github.com/cnlohr/espusbYT link because apparently you dont click links or understand text

author: Red Squirrel
author_rank: Super Contributor
timestamp: October 16, 2016, 03:21:21 am
content: I don't want something wireless, though I could probably look at how easy it would be to incorporate the chips directly somehow. I will add that as a potential option.