Minecraft Infested Stone?

I mined at level 12 and suddenly I found contaminated stones (i.e. The ones from which endermites come out). Does that mean that I'm near the Ender Portal?

Yes, I think so. As you know, they are not far from Ender Portals. Just make yourself an Ender Eye and find out.

So with me they also showed up without an end portal in the vicinity. It's not that often, but I think they can spawn at a certain depth. But I'm not sure.

Either an End Portal nearby or just in an Extreme Hills Biome, that's where they are.

Doesn't have to be, usually it's just a sign that you're under a mountain biome.

The little animals are called silver fish.

And nope, those infected stones can also be found under mountain biomes.