To beat the Green Ruins, you need to find the particular staircase on each floor which leads down to the next floor (this is not the same as the shortcut stairs in the bottom-right corner of each floor, but you can unlock the doors from the outside each time you reach a new floor so you can use the shortcut to return quickly to that floor when you explore the dungeon again later). Keep searching each floor for the stairs and continuing down until you reach B4F. Then just examine the large door, accept, and proceed to the boss fight. Once you win it, congrats! You've beaten the Green Ruins and can progress with the story. (Check the FAQ/Walkthrough of the game for more precise directions.)
"I also need silver to get rid of these annoying rocks on my farm."
Once you open up the Lava Ruins (after you beat Green Ruins) you just have to mine the rocks there and you should be able to find silver fairly easily. Until then, if you run all the way to the right of Whale Island Flank to the area with the large tree and rolling fruits, if you cut the vines to the north and go through, you'll find some purple rocks which may yield silver every once in a while. To increase your chance of getting good ores from rocks, just charge your hammer swing when breaking them.
The earliest I found Silver was in the Lava ruins on the second floor. There is a hallway with two golems you can just run past and ignore and then there's a bombpot next to a few rocks and the enemy spawn is in the top left corner and will make a duck near it and a fire elemental thing at the opposite corner. You can usually tag the bombpot with a spear and not activate the enemies until after the rocks are blown up or you can pound with a hammer. I consistently got one silver and two iron scraps there...
Then I found out if you charge the hammer up before you hit the rocks it can change what you get from them and I ended up getting a ruby before moving on deeper (and in the last room before the boss were some rocks and one spawned gold, but the rocks weren't there when I came back to beat the boss...)
As for going to the rolling fruit area of the whale island...geez...I just opened that part right before beating the lava ruins. I'd been milking the first three floors of the Lava ruins while mixing time between the farm and the Whale. I wonder if the earlier areas would spit out some silver with a charged bash? Maybe not. I don't feel like going back to test, either.