It could be exactly that hun, leo's do lose their appetite before shedding so wait and see how he is after that. If he eats his skin and poops following i'm sure he will be fine.
If his poop looks normal I wouldn't worry yet. I know how panicky it can be at any sign of abnormality but I'm sure he will be ok

Have kept quite a few leo's now and raised hatchlings so my nerves have been up and down!
Also keep him off the meal worms until he is a little older - they won't help anything. Try giving him locusts or black crickets as they have the most nutritional value and will make sure he grows up to eb a abig healthy leo!
Oh also do you have a wet box set up to help with shedding - it's really important to make sure no skin stayes on his toes or eyes!
Han x