Sorry a bit late but just picked this up.
Yes river fishing close to and in Edinburgh with the Water of Leith a good wee river and permits are free and available for Edinburgh Angling Centre. Wild browns and grayling.
There's also the River Almond which is near the airport, good wee stream.
On the outskirts of the city are both Esks, North and South which join near Mussleburgh. Again wild trout and grayling and both cracking wee streams.
Lots more options within an hours drive.