Today, high brightness and highly relativistic electron beams are generated by circular or linear accelerators (LINAC) typically operating with 1-3 GHz accelerating frequencies and approaches towards X-band frequencies in the 10 GHz range are maturing. The achievable accelerating gradients are limited by field emission from cavity walls or pulsed heating to several tens of MV/m in the case of low frequencies and up to 100 MV/m in the case of X- band frequencies. Moving up in frequency to the THz range, here hundreds of GHz, experimentally confirmed scaling laws predict the realization of few-hundred MV/m to 1 GV/m accelerating fields in LINACs and guns, respectively. The high field strength and field gradients enable direct generation of single femtosecond electron bunches with substantial charge in the pC-range from very compact devices. We will discuss highly efficient laser based THz generation by optical rectification and first results towards THz based electron guns, LINACs and beam manipulations with the final goal to construct a hard-X-ray source producing attosecond pulses.