Help me make Elves different from Magic-Users + Random table time

I don't like the idea that Elves are basically the same with Magic-Users.

In a nutshell M-Us learn spells by research.

I want Elven magic to be natural but chaotic in nature. Elves cannot learn spells from the book thus they don't need spell books. Elves know magic "from inside" as they are chaotic beings.

1st level Elf character knows as many spells as appropriate and can memorize the spells. Elf cannot learn new spells before he levels up. That's because Elves advance as a character and that's when they learn new "hidden" abilities.

Elf known spells are randomly determined. As Elf magic is chaotic they cannot know or decide what spells they will learn or know. So when creating an Elf character randomly roll the spells that chaotic individual knows. Also when Elf levels up randomly determine what spells his advances chaotic character now can cast.

But there's a problem here. As Elves only learn new spells by growing as a person (leveling as a character)they will have less spells to use compared to their collection in spell book. So how to fix this?

In addition Elves can brew potions, use scrolls, wands and staffs as M-U. Only the spell research part and copying spells to their spell book doesn't happen. So how to replace the spell book to keep the balance but make the chaotic nature of Elven innate spell skills more obivous?

Elves can memorize spells as their level allows, but how many spells Elf can know at each level? I was thinking that Elf finds out new spell powers inside him when he levels up as many as he can memorize, but isn't this downgrading Elves known spells?

Example (I pull this out of my hat because I don't have reference to actual spell progression at hand):
Elf now can memorize at his current level two 1st level spells and one second level spell. These spells are randomly determined what he knows. Naturally it's up to player what spells the Elf will memorize (one of each or two 1st level spell slots with the same spells).

Next level Elf can memorize three 1st level spells and two 2nd level spells (pulled that out of my arse). So player randomly rolls extra first and extra second levels pells the Elf character invokes (his chaotic nature developes). Now the character can memorize three 1st level spells and two 2nd level spells and knows as many spells...

But that's the problem I got. The Elf is stuck to the amount of spells equal what he can memorize compared to spell book where can be lots and lots of spells. So how to fix this screwery?

One option I was thinking is that Elf can try to cast a spell without forgetting it. But because Elven magic is chaotic, there's a risk doing this. So Elf can either:
a) cast and forget
b) cast without forgetting with a risk

But what's the risk? The risk is, that the spell might backfire. The reason basically is, that it is much safer to release the spell power and forget it than release the spell power trying to maintain its force in his mind.

How this risk works? There are few options for this:
a) Save versus magic (or equivalent) - spell level to die roll (example. Elf's save vs magic is 15 and he tries to cast a spell without forgetting it of level 3. So he must save against 12
b) Change is 1 in 20 + spell level that the spell backfires (1-4 if 3rd level spell)
b) Roll under Intelligence with d20 + spell level

(In my opinion save versus magic is most logical and tweaks the mechanics the less).

And the backfire? There's two options:
a) the spell "explodes" on Elf before its power is released (Elf casts sleep on self, hurts as the fireball explodes inside him etc.) No save allowed (because basically he already saved)
b) random table time! With effects from spell effect on self to you died (Raggi style random table result). The results should be chaotic in nature as Elven magic is chaotic. So there could be anything from shooting ye friends or casting buffing spells on monsters. Or reversed effects. Or mutations on Elf (Hideous form, loose permanently CHA) or in worst case let's make some summoning rolls (because chaos).

The a option is simple the b option is more interesting.

BONUS

Agree or not with my Elf modification or if you don't have anything to say about it you can still give a hand here! Give me some random spell failure effects! Let's make a sweet spell backfire table here!

tl;dr
If Elves cannot research and study and find etc. new spells nor own or can use their personal spell book, how their spell advancement works?

Oh, and maybe I should check how D&D 3e Sorcerer class worked... There might be some ideas. Maybe LotFP Elf (and why not OSR in general) could be Fighter/Sorcerer instead of Fighter/M-U.