When you're using the Lucky Charm you may also run into the Lead Guardsman who you can use the Charm on a few times to get to the switch.
If you chose the correct switch and the correct response, you will distract the guard, and then hit the switch. Depending on the item you choose to start out with you may get some additional characters showing in the cutscenes. The one with the Snares is particually funny.
If you did it correctly, you will get a message saying "The <number> switch has been deactivated."
If you did it incorrectly, you might still be able to hit the switch, but you will get no message.
Or you will just fail to distract and the guard will catch Luto or Mildeo.
Once you finish them all, you get a message saying "You hear a noise coming from the Temple of the Goddess."
Go back to the Temple and check the ??? to get a final cutscene and to complete the quest.
Note
If you get caught, you can free the person caught by going to the Hume Male in the Craftsman's Eatery and then the Galka child who you got the switch order from near the blacksmith guild, Folzen I believe.