Sending Sensor data with Video on same Ethernet

Any miscellaneous pieces and parts that don't fit in the above forums.
rovdude
Posts: 83
Joined: Sep 17th, 2012, 10:28 pm
Location: Baltimore, USA

Sending Sensor data with Video on same Ethernet

Post by rovdude »

So after a lot of research, I have found I can use the Raspberry Pi for 1080p 30FPS Full HD video, as well it can take in the sensor data from the Arduino. The sensor data should be an array of 13 or 14 doubles, and sent every 30th or 15th frame. Does anyone know of a way to send the video feed and sensor data together?. I will be sending it through ethernet initially in the ROV, to be converted to Fiber. I have three main ideas:

1) Have the Arduino send it's sensor data to the Pi, and have the Pi send the video, and then the data goes between frames.

2) Have an ethernet cable from both the Pi and the Arduino soldered to the transceiver, have the Pi send video, and when the Pi is not transmitting, have the Arduino send its data.

3) Use Processing on my laptop and Processing on my Raspberry Pi, and use the NETWORK library to set up a server/client way of sending data.
rossrov
Posts: 383
Joined: Feb 28th, 2013, 5:01 pm
Location: Australia

Re: Sending Sensor data with Video on same Ethernet

Post by rossrov »

Hi Rovdude. What is the video delay or latency of the Pi? If acceptable, can the Pi communicate with the Arduino via I2C or some other means? That Arduino Yun you mentioned on the Fibre Optic thread appears to have Pi camera and Arduino features on the same board. Have not investigated much, and know very little of the Pi, for that matter, however, the Pi does not look anywhere near as easy to use for PWM and other IO as say an Arduino uno, otherwise I would suggest ditching the Arduino and use the Pi for camera and IO.
If you look at viewtopic.php?f=15&t=997&start=10#p7879 you will see that I am using an Ethernet switch with Ethernet Arduino and a cheap IP camera. The testing that I did with that combination was using two browser windows, one sending HTTP requests for getting data back from the Arduino to the PC browser window, and the other window for the video.
Now I am planning on using UDP and Processing for the data and control, and a browser window for the camera, same hardware as above, though have not tested this setup yet.

Ross
rovdude
Posts: 83
Joined: Sep 17th, 2012, 10:28 pm
Location: Baltimore, USA

Re: Sending Sensor data with Video on same Ethernet

Post by rovdude »

The video latency is reported to be low, ~.4s, but I cannot verify that because I currently do not have the camera module :roll: . The Pi does have an I2C that will work with Arduino, and there are multiple projects out there that have used them together. The Pi is harder to use IO/PWM on, because it doesn't run like a microcontroller. A microcontroller like Arduino usually runs code in a loop, while with a Pi you are writing a normal computer program.

i'm still getting used to the Pi too. But it's literally a miniature computer that runs Linux, has a desktop mode, runs finished software, and you can run programs on it in almost any common language. It COULD probably work with most of my sensors, but the main reason I am keeping an Arduino Due down there (or another 3.3V Arduino) is it can supply enough 3.3V power for a fiber optic transceiver. The Pi just can't give that ouput, and while techinically I could put a voltage regulator circuit in direct off the batteries to power the tranceiver so I wouldn't need the Arduino, it seems safer to me to use the Arduino's regulated voltage output.

What you were mentioning on that other thread sounds great. But what quality was it? I'm trying to do 1080P 30fps... which is quite demanding :lol: . The pi camera works similiarly, but only requiring one window. I am planning to use Processing on top for everything. Have not fully investigated, but THIS library was made specifically for IP cameras. Through that was hopefully going to display both data and video.

Another idea for sending all the data down one ethernet was to have the pi program send the video when prompted like a normal IP camera, and tell it to send the sensor data every half or full second. See if the Pi can handle it by itself, and see if processing can decode it without any issues without any detailed coding of how to manually handle the data.
fryslan76
Posts: 290
Joined: Dec 18th, 2012, 4:52 pm
Location: Netherlands

Re: Sending Sensor data with Video on same Ethernet

Post by fryslan76 »

Hi,

I posted a smal bit about I2C in this feed: viewtopic.php?f=41&t=1386

By using the Arduino as a slave which replies on requests and does the normal tasks like driving the motors and reading the senors, using the PI for video should be no issue. Just don't try it with the model A, that one is underpowered for this. Video takes to much as far as my experiments go.

KR,

Fryslan76
rossrov
Posts: 383
Joined: Feb 28th, 2013, 5:01 pm
Location: Australia

Re: Sending Sensor data with Video on same Ethernet

Post by rossrov »

Rovdude: 640 X 480, and I'm happy with the frame rate and quality. Higher resolution will place more demands on the tether. Anyway, I guess what you are describing is similar to how "analog" TV teletext is sent and decoded. There is non-image data sent along with the image information in H264 and others so sure, if you really wanted to put sensor data in place of the subtitles and timestamps etc then no doubt this would be possible. I fear though it would need alot of work.
A 5 port switch costs about $10 and doesn't take up much space if you remove the case and connectors, trim down the board (literally, with an angle grinder :D ) and solder direct to the board where the connectors were. Topside, if Processing will do the video OK, you put the respective IP addresses of the IP camera/Pi with cam and Arduino Ethernet shield in your code and hopefully that's all there is to it. The switch will do the rest of the work for you, putting the sensor packets between the video frame packets.
I've had a look at some of the Processing video streaming stuff but do not like using libraries that are not in common use, hence sticking with the browser for camera display. Would be nice though to have the one application for both video and sensors/control.
Fryslan76: Did you figure out a way of getting the video and sensor data up using just the one IP address (the Pi's) in the ROV? Good mention about the model A.
rovdude
Posts: 83
Joined: Sep 17th, 2012, 10:28 pm
Location: Baltimore, USA

Re: Sending Sensor data with Video on same Ethernet

Post by rovdude »

rossrov: Hmmm... that ethernet switch actually sounds like the way to go. I don't know much about them, but am I correct by saying it takes multiple ethernet inputs and sends it out one (providing there's enough bandwidth)? And yeah the teletext is pretty much what I was going originally thinking of. Didn't think about it like that, might help with research
fryslan76
Posts: 290
Joined: Dec 18th, 2012, 4:52 pm
Location: Netherlands

Re: Sending Sensor data with Video on same Ethernet

Post by fryslan76 »

@rossrov, I placed the arduino as I2C slave for the PI, so I was only using the network connection of the PI (one IP). Had a long ethernet cable to a router in my house and was watching the video and arduino sketch through an X windows forwarder on my laptop. The usb camera I was using was an MPEG stream, which gave about 5 fps when I was lucky. So nice for a bench test but to slow for in the water.

The powerethernet connectors I found have a build in hub for three ports so I can now connect an IP Camera and an Arduino with network shield and even an PI if I want. :)
nomore
Posts: 6
Joined: Jul 2nd, 2012, 6:18 am

Re: Sending Sensor data with Video on same Ethernet

Post by nomore »

How about using the audio channel for telemetry? This is quite common in FPV community :)

Heres an arduino schetch that uses this approach: http://diydrones.com/profiles/blogs/ope ... -telemetry
rossrov
Posts: 383
Joined: Feb 28th, 2013, 5:01 pm
Location: Australia

Re: Sending Sensor data with Video on same Ethernet

Post by rossrov »

Yes rovdude, one PC and one Ethernet cable, multiple Ethernet devices in the ROV. The Ethernet over Power that Fryslan76 is using already has multiple device capability with it's 3 sockets, but that's not going to help you with fibre later down the track, nor will it help me unless I get the EoP units from the UK or Europe, as I can only find the single-socket versions in Australia :( . I was in Aldi a few days ago, Fryslan76, actually there buying food of all things, and must confess to having a quick look for some adapters :? . They were selling them apparently, last year though. I Googled X-windows forwarder but it looks a bit outside the scope of things for me.
Nomore, I found that switching the mic on in my IP camera increased video latency a surprising amount. This was one of the first tests I did with the camera. Whether the delay was in the camera or in my small slow PC I do not know, but discounted the audio path as a possibily for data based on that. Would still consider using the audio channel for data in an analog RF-based tether - thanks for link
a_shorething
Posts: 289
Joined: Sep 10th, 2013, 5:26 pm
Location: New Jersey Shore

Re: Sending Sensor data with Video on same Ethernet

Post by a_shorething »

nomore wrote:How about using the audio channel for telemetry? This is quite common in FPV community :)

Heres an arduino schetch that uses this approach: http://diydrones.com/profiles/blogs/ope ... -telemetry
Wow, that's cool stuff!

I want to check this out. Right now my plan is similar to rossrov's plan: switch either onboard or on the surface (ethernet only requires 2 pair, Cat-5 has 4 pair).

This looks a little more involved than what I had in mind, but very cool.
Post Reply