VGA framegrabbing with TVP7002

Okay, maybe it’s finally time to write something about this project I’ve been working on for quite a while. It has finally reached a stage where it’s quite cool!

The story is that for some strange reason, I’m showing great interest in implementing a network based KVM for managing my home server remotely. To do this, I need to be able to capture VGA output from a PC, and transfer it over some medium (ethernet) to another PC.

What I want to do, is basically this:

plan-450

I’ve had the frame capturing part of the project working for a while now, and just recently, I’ve managed to get ethernet working.

Currently I’ve managed to do this using the following devices:

  • TI TVP7002 video digitizing chip
  • NXP LPC1768 32-bit ARM Cortex-M3 microcontroller from NXP
  • Altera MaxII EPM240 CPLD
  • ISSI IS61LV5128AL 512Kx8 SRAM
  • Microchip enc28j60 ethernet controller
Project in full operation

The board is ugly, full of kludges and completely temporary.

An online demo of the web interface can be seen here. If the image is not changing, that probably means my board is off-line, and only a cached image can be seen.

I’ll write a little bit of details below. If this gains any attention, I would be happy to write in more detail about various aspects of the project.

TVP7002

The project got a nice boost when I found TVP7002 on TI’s site. It’s a video digitizing chip, featuring 3x ADC (one for each color channel) and a PLL system for generating the pixel clock from the HSYNC input. Compared to alternatives, this seemed to be quite nicely priced, and well available. In fact, TI was kind enough to provide me with free samples!

Without this chip, or a chip like this, I would have no hope in implementing a project like this. Since I don’t really know much about electronics, it’s very helpful to be able to hide all the scary and magical stuff away.

Here’s a simplification of what the chip does in my project:

tvp7002 operation super-simplified

So instead of having to deal with the analog signals myself, or having to generate an appropriate pixel clock, the TVP7002 does this for me. After setting up the chip appropriately, I’m left with easy to understand digital signals.

To read this image data, I basically need to keep up with the sync signals to know when a frame or a line starts. On each pixel clock, I can read the color channels for pixel color data. This is already well familiar to me from my experiments with the OV7670 camera (see my blog entries here and here).

FPGA / CPLD

The speed at which I must read the pixel data is pretty high – 25.175MHz for standard VGA resolution. It’s not really easy to use a standard microcontroller for this, so an FPGA is a better choice. In theory, the data could be clocked directly onto some FIFO RAM, like done on the ov7670 module (see here). However, I did not really bother looking for suitable RAM like that, and decided to process the signals myself instead.

Some captured data when things were not that perfect yet

Some captured data when things were not that perfect yet

At first, I set out to read one pixel at a precise location. This would be enough to prove that the setup can work. This was successful, and I was able to draw a full frame by reading one pixel at a time – each pixel from a different frame. Transferring the image to a PC took a very very long time, depending on how I packed the data. At this point, I was not using any external RAM yet.

Later, adding SRAM to the project was relatively easy, using this simple Verilog SRAM controller as a base. The SRAM access times are determined by the CPLD clock – which is also the pixel clock – normally 25.175MHz for standard VGA resolution. Read operations also use negative edges of the clock.

Originally, I used an Altera Cyclone II with a cheap development board from eBay, but since the required logic is very simple, I later switched to a CPLD. The transition was pretty easy, but the logic had to be simplified by replacing SPI with simple IO signals.

Microcontroller – LPC1768

The choice of microcontroller is not really a very critical part of this project. When a video frame is captured to SRAM using the CPLD and TVP7002, transferring a full, beautiful image to the PC can be done at any speed, using any desired protocol. That said, I will not be making an Arduino shield of my project!

LPC1768

Currently I’m clocking the data from the SRAM to the microcontroller at around 400kHz. I could go faster (if the MCU can keep up), or slower (if the MCU can’t keep up). Although I’ve successfully utilized DMA for this, it didn’t seem to be worth the effort at the time, and right now I’m using simple GPIO routines instead. There’s definitely room for optimization here, which I will look into later for improving the frame rate.

The choice of microcontroller may be important for some other things though; I wish to use ethernet for the data transmission – and LPC1768 has Ethernet MAC built-in. However, currently I’m using enc28j60, which could be used on any microcontroller.

In future this project would benefit from having USB support for implementing an HID keyboard, so if LPC1768/69 turns out to be overkill, I could perhaps use something like LPC11U14 instead. For now I’ll stick with the LPC1768.

ENC28J60

Although LPC1768 has a built-in Ethernet MAC, I decided to start my Ethernet experiments with Microchip ENC28J60 instead. This was simply because ENC28J60 seemed more approachable with many working examples available for inspection.

I started by porting over this stellaris-enc28j60-booster project, which uses the uIP stack. This worked fine, but The download rates I got with uIP were pretty slow. That was to be expected, since uIP is not really designed for high throughput.

Using the same ENC28J60 driver as a base, I switched the stack to lwIP. Porting the stack was kind of straightforward, as described by this document, but certainly not easy for me. Later, after I had gotten the lwIP stack working, I realized the same Stellaris ENC28J60 project also had a lwIP branch here, but it was too late.

With this stack, my top transfer speeds are around 300KB/s, which is pretty nice. Due to slowdowns from the image transfer, I’m currently transferring stuff at around 100KB/s.

enc28j60-fails

I think the lwIP stack is probably working perfectly, but the ENC28J60 is still giving me a headache. At the moment, it could be running 24 hours without a problem – and then suddenly stop working for no good reason. After multiple resets and ritual sacrifices, it will resume normal operation again. It could be some problem with the breakout board I’m using, and I’ll try to not get too anxious over it for now.

The future, and stuff

grub-grabbed

This writeup is rather short and without much detail, but each part of this project has been a huge effort. Later I do hope to write in more detail about some things.

When I started working on this, I knew every little about electronics. Now that I’ve gotten this far, I still know very little about electronics, but a tiny bit more about digital electronics. My background as a code monkey has certainly helped a lot in understanding this stuff.

Right now, there’s plenty of stuff to tweak and improve in the current design. At the moment, things are hardcoded in, and the board just assumes the input is always 640×480@60. There’s a lot I can do regarding that by just writing more code.

However, it’s also time to start planning some hardware improvements. First thing in mind is a new board with the design and manufacturing errors fixed. Another thing is probably to start looking into the LPC1768 built-in MAC and the LAN8720 PHY.

I don’t know what I will do about with memory yet, but obviously more is needed if I want to support bigger resolutions. Faster pixel clock rates will also require some consideration.

And USB support.. some day…

Code

I’ve dumped the microcontroller and Verilog code on github here. I don’t intend this code to be used by anyone as-is, but maybe some bits of it could be useful to someone.

Schematic is also available here: grabor v0.1 schematic

Links

Here are some links I found useful during this project:

27 thoughts on “VGA framegrabbing with TVP7002

    1. desaster Post author

      Hi VGA Switch!

      Thanks for your comment!

      There will certainly be more to come, as I’m slowly getting back into the project. I am considering writing something more detailed about the project in it’s current state as well, but not yet sure what exactly.

      Reply
  1. VGA Switch

    Thanks desaster! It would be really appreciable to get more information about it as soon you get back to the project. And more detailed information would really help me a lot too.

    Reply
  2. claude

    Great — I want to buy a dozen, maybe as many as 100+ in time. Must be cheap-ish (eg: < $40 ea (or UK £30~)). Can you help? I may consider assembly in the UK, but would prefer you built the boards and tested. We would provide the cases here.

    We just want a web interface too so the way you have built it appears to be perfect.

    Purpose: very low cost "slow scan" KVM over IP — we can interrogate the image with a raspberry pi, recompress, do the keyboard separately, etc. However, the project won't be viable if the costs can not be sustained.

    My only question is — does it support different video rates up to 1024 x 768? Does it auto change? (this is so we can take a snapshot of the CMOS bootup text etc). Can it refresh once every three seconds?

    Thanks, Clyde

    Reply
    1. desaster Post author

      It’s great to see interest in this project! Unfortunately it’s still in a very prototype state, and being a hobby project I have no plans to make it into a product.

      The basic principle of the frame capture can support any resolution, but will need higher speeds and bigger RAM. Not a priority for me yet.

      The speed of capturing the image in RAM can be very fast, but the limiting factor is the actual transfer of the image via network (or other medium), as well as the communication speed of the microcontroller (or raspi in your case).

      My next plan with this project is to try and improve the ethernet functionality with lpc1769’s ethernet support, then build a better power supply, and make a new board out of it all.

      It’s nice to hear someone else sees the value of a low cost KVM :) Unfortunately I can’t be of help in providing you with one…

      Reply
  3. Claude

    I gotta say as a PCB designer myself, that’s quite a project to undertake for a hobby.. You’ve obviously got a better brain than me.

    I did note it is just 640 x 480 and should have asked “will it” be able to auto change? But anyway, I completely understand that you don’t want to be under pressure. And yes, as a low cost IP KVM, it could have a wide audience. My max potential is 200 units over 12 months. I’m sure others have more potential. If you do decide to go into production or sell the plans & s/w under license or whatever, let me know (email me on the hidden address).

    At the moment, I’ll probably use a VGA to Video convertor and I’m sure there must be a Raspi video input adaptor. We’re flexible! (It’s just a remote front line KVM check — we need to be able to read at 800×600 mostly .. even if blurred .. that’s all).

    The very best of luck! Ps: Many thanks for sorting the name too. “Claude”.

    Reply
    1. desaster Post author

      I’d love to hear more about your project. Personally I haven’t invested much time in my own KVM lately, but hopefully I will get back on track sooner or later.

      Reply
  4. John

    A nice project.
    And is it possible to “connect a keyboard”. I mean there will be a login screen where I must fill in ID and Password on a server.
    Is it possible to insert those IDs from the remote computer? In other words is there communication in both direction?

    Reply
    1. desaster Post author

      I have certainly planned to implement USB keyboard input as well. However there’s still much to do in other areas, so it’s not a priority.

      Reply
  5. Pingback: .NET i jiné ... : Odkazy z prohlížeče – 24.2.2014

  6. Steven

    Hi

    Nice article, hope to see some more about this project.

    Could you tell me where you got the TVP7002. TI does not have samples of the device and after a quick web search I only found one supplier in the Far East. I’m in Cape Town South Africa.

    Regards,
    Steven

    Reply
    1. desaster Post author

      the tvp7002 chip is now considered obsolete by TI, so getting samples is no longer possible. I guess they are giving up on VGA in general, and providing solutions for HDMI and stuff instead. Digikey still seems to have a few left.

      Reply
    1. desaster Post author

      the blog has a link to the schematic, if that helps. it has some issues that needed fixing later, but should give you the general idea.

      Reply
  7. mehrdad

    Dear friend
    At first thanks a lot for sharing your knowledge and expriences.
    But I have some questions.

    can you tell me what environment have you used for creating your project?
    did you use keil or coocoox or any other ide ?as

    I wanted to compile your project but I faced some errors.
    please guide me
    Thanks a lot

    Reply
    1. desaster Post author

      I don’t use an IDE for my MCU projects. This project was built using the Sourcery CodeBench gcc toolchain; arm-none-eabi-gcc (Sourcery CodeBench Lite 2011.09-69) 4.6.1.

      The source code structure is based on Freddie Chopin’s ARM example projects

      Reply
  8. mukhe

    That’s some really nice soldering job you have done there. how about making a video on how you did it? i did mine and it looks as horrible as you can possibly imagine

    Reply
  9. Miguel Encinas

    good day,

    we are very interested in your project can you provide us with a mail account or something so we can chat about your project ?

    Reply
  10. Alex

    Hello,

    I notice some of the chips are now obsolete, namly the TVP7002.

    Do you have any recomendations for alternatives?

    I want to achieve the same thing as you set out to achieve here, with the goal of making an open source IP KVM.

    Many thanks,

    Alex

    Reply
    1. desaster Post author

      Hi Alex,

      Indeed the tvp7002 has been marked as obsolete for a while now. I think that may be because they are considering VGA obsolete. Perhaps there will still be stock available for it in the future.

      I don’t know of any alternatives like this. I liked tvp7002 because it was quite cheap compared to some similar chips, and it did a lot of the heavy lifting for me.

      However, for an open source solution, perhaps a more generic approach would be better, where the design would not be stuck to a integrated solution such as tvp7002.

      Although I haven’t worked on this project for a long time now, the topic is still very interesting to me. I’d love to hear more from you if you make progress with your project!

      Reply
      1. Alex

        Shame, as many servers still running (or even new) are VGA.

        I’ll see if I can order up any old stock – even if it’s enough to hack together a prototype. That experience alone would probably be a great education.

        I’ll be sure to drop you a line when I make some progress!.

        Reply
  11. Pingback: Rasky update: choosing how to connect the CPLD to RaspberryPI | Nexlab

  12. Monah Tuk

    Hi!
    Do you work only with predefined input resoultion (VGA, 640×480) or you implement algorithm for PLL autoconfiguration? In other word: How are you configure TVP7002?

    Reply
    1. desaster Post author

      This project was in such an early stage, that I only used a pre-defined configuration of 640×480. Automatic configuration would have been a future step, if I had continued working on the project.

      Reply

Leave a Reply

Your email address will not be published. Required fields are marked *