Gamma-Ray Bursts (GRB) are transient sources, emitting radiation on a wide range of wavelengths, from radio to soft gamma-rays. The existence of radiation emitted at even larger frequencies, in the GeV domain, was discovered (in an handful of GRBs) more than 20 years ago by EGRET, on board the CGRO satellite. This radiation can now be studied in more detail thanks to the Large Area Telescope (LAT) onboard the Fermi satellite, launched in 2008. So far, the LAT (40 MeV - 300 GeV) has detected GeV radiation from about 100 GRBs, around 10% of the observed population. For the brightest events, the temporal and spectral properties of this emission can be investigated. Its origin and its relation to emission detected at lower frequencies is still uncertain. In this talk, I review the observations and theoretical advances towards the interpretation of GeV radiation from GRBs. I will outline how the study of this emission is helping us in improving our (still poor) understanding of the GRB phenomenon. In particular, the detection and study of the high-energy radiation is allowing us to learn about the still unclear radiation mechanisms at work in GRBs, their efficiency, particle acceleration and amplification of magnetic fields in ultra-relativistic collisionless shocks. Finally, I will discuss the prospects for GRB detection in the GeV-TeV domain with the Cherenkov Telescope Array (CTA).