I saw an article last year about a Neural Network creating new lyrics for Christmas songs and wanted to give it a go myself.
I've used an LSTM in Keras, and you can too...I've just used the text generation code from the Keras team (located here).
I didn't change much in the code apart from changing the input corpus of text from Nietzsche to a text file full of Christmas lyrics which I compiled from scouring the web. Remember, for a task like this with an LSTM you really want a lot of data, so collect as much as you possibly can as you won't get many sensible words being formed with limited data. The more the merrier so to speak.
Anyway, I ran it for around 100 epochs and then looked through and picked out some half decent (maybe an optimistic half...but better than some of the gibberish that was coming out) lyrics, see below:
It has certainly mixed up some concepts, but I like it! I'm not quite sure what it was attempting with "dreitids"...
I'm off to collect more song lyrics for my corpus and try again. Have fun, and Merry Christmas!