The Foredt, Castle, etc are SEPARATE locations. You need to look at your quest tab. I believe there's a quest to VISIT the Challenge Mines before a new location opens for you. Or perhaps you need to talk to the Fairy Godmother.
No need to keep entering Merlin's Tent until you've completed ALL the 8 the Main Event Story mines, as each mine contains only one bell. Then once you have all the belks, you go back to the Swamp mine, click on Merlin to give him the bells. He'll then move so you can get the key.