So I realized I waste a ton of time clearing out feet slots after each grit of sandpaper. I then tried to cut very thin strips o maple veneer to ill the slots but that failed miserably. I went to the bathroom and saw my dental floss on the sink. I really should use it more but I realized I could use this to fill the slots instead. It worked beautifully and by the end I had a nice bag of ebony dust to fill the fret slots at the end.

Check out how the fretboard gets smoother and shinier as I go from 220 to 320 to 400 grit. Afterwards I just hit it with synthetic 0000 steel wool.