How do I replace this odd outdoor outlet and light setup?

UPDATE:

First, thank you all for the responses, I really appreciate it. I first tried to drill through the screws but I couldn't make it work; they were encased in thick metal, almost as if thick metal anchors were used. And those weren't budging, at least not with the drill bits I used.

With that option off the table I began to cut the box out of the caulk, and boy was that a task. It was floating entirely in caulk, which in turn was attached to the siding frame. (it didn't actually seem like caulk, much more tough and rubbery, but I don't know what it was). Behind it was just a hole through the wall/insulation with a single romex coming through (yes I cut the breaker). Of course the attached conduit also wouldn't budge so I had to twist it off at the switch box.

So I'm left with the big hole, without siding to use as a flat surface, and without a stud or wall to attach a box. I've taped an improvised cover on it for now and will turn back to it soon. Wish I hadn't opened that box!

– Original:

TL/DR; I need to replace an exterior electrical outlet box that is also supplying power to an exterior floodlight through surface mounted conduit. The box is partially counter sunk into the siding and seems to be caulked in place. What do I do? (this is a bit long but the pics should help)

I started to unscrew the outlet from its box outside and it popped off along with the tops of the corroded screws, leaving the rest of the screws in place and preventing the use of replacement screws to get the outlet back on the box.

Replacing the box looks more complicated than I expected. Power runs to the outlet and then up through a conduit up against the outside of the wall. It stops to connect to a switch in the adjacent room and continues on to the light fixture.

Here's where it gets a bit weird. First, there is a hole in the wall for the box (framed by siding) but it is bigger than the standard single gang at 5.5 high and 3.5 wide. Second, because the conduit needs to run outside the wall through a punch hole in the box, the box is partially in (3.75 inches) and partially out (1 inch).

That leaves an awkwardly floating box surrounded by sloppy siding-colored caulk (or something). I could just replace the box, but I'd have to cut it out of that caulk and I don't have or know that color. I can't replace it with a flush outlet regardless of the caulk because there needs to be an entry point for the cable going to the light. Adding a big cover that reaches to the siding frame has the same problem.

On that wire to the light, I don't know that I can fish it inside and back out to the light. I could probably live without the light, or maybe get a light that is solar or has a plug that I can run down and into a cover. That still leaves me with the weird box setup.

Any thoughts would be greatly appreciated.