You wanna get into Linux, cheap? Here you go, all you need is a monitor and a mouse. It already comes with Linux installed and for under $200!
Although I'd ditch the Ubuntu Linux installed on it and go with something like PCLinuxOS. I've tried both and PCLinuxOS was a breeze compared to Ubuntu.
Or better yet, build your own! Older PC parts are very reasonably priced on EBay and will work as you don't need a lot of oomph! to make most Linux distributions work well.