If you know their name and they are in the same district as you then you can just type the person's name as mentioned above. But if you're in different districts one of you will have to jump to the district.
I find the easiest way (if they are talking in local, would work for anyone you can easily whisper) is to click their name like you are going to whisper them. Then highlight and hit ctrl+c then ctrl+v in the invite box.