Just got Reaper, preapre for newb questions!

You need to download the Reaper pdf helpfile. It is awesome. In fact I bought the book spiral bound from Lulu. The keyword to search for is routing. There is a matrix option that is quick to adjust and then you also need to assign your inputs within the session itself. The keyboard will not automatically be assigned a track. Now, if you have the inputs assigned correctly you will still need to assign an output to the master fader. Assigning the fader to something not present or active is as good as no assignment. I don't know the deep details of Asio4All so I won't be much help there.

Section 2.2 page 48 of the guide to which you linked. Also section 2.3 for the matrix. Additionally section 1.12 speaks to setting up the ASIO or in your case fake ASIO driver or whatever you are using-WDM device etc.

You aren't out of depth. I promise. It is just unfamiliar to you. Windows or Mac kindly automates basic functions however all DAW programs require actual routing configuration of one type or another. If you can use a physical mixer and gozinta and gozouta then this is the same only with GUI screens instead. Think in terms of wire. Go to the other chapter to which I linked. It talks about setting up an ASIO driver. In your case you may have to do addtional setup within the Asio4All (A4A) GUI (graphical user interface) because it really isn't a proper driver. Within that gui you can assign in and out as well.

Good. It will work with A4A as well but I don't know what to tell you regarding A4A settings as I don't use that. At any rate, eventually you will probably grow into a proper audio interface and then you will have a prope asio driver.

Correct. The Scarlett will come with an proper ASIO driver. At that time you will have to learn some configuring but not nearly the headache A4A would have been.