If you pick up a copy of The Art of Keeping Snakes by Philippe de Vosjoli, there's quite a bit in there about "bioactive substrate systems", which is more or less what you're talking about there.

It is possible to do. However, sterile soil is not possible to have but that doesn't matter, because you definitely want bacteria in it. If you have the right type of bacteria in the substrate, they'll take care of breaking down the feces. Unless you are flooding the enclosure with feces or carcasses, there shouldn't be a problem with harmful bacteria and the amount of nutrients available in the environment will limit the amount of bacteria and competition would favour those specialized in that type of environment.

As with aquariums, the bigger the enclosures are, the easier they should be to run and maintain. The biggest problem would probably be too much humidity, which would cause problems with mold and increase the risk of various infections.

As some of you may know, I ran a three month (IIRC) test last winter in a relatively small (70 litre) enclosure, that housed my viperine snakes. The substrate was a mix of peat moss, sand and leaves and the plants I used were pea plants (for practical reasons). The surface was covered with a layer of dried leaves. During this time, I made a few observations:
- The feces were broken down very quickly. They'd disappear virtually over night.
- The plants sucked all the excess moisture out of the soil. Presumably, they did the same with the nutrients.
- The roots made the substrate very compact. Unlike with ordinary substrates, there was almost no loose material that could have been accidentally ingested. When I ended the experiment, the substrate was lifted out of the enclosure in one solid piece.
- There was virtually no dust, even when I let it dry out towards the end.
- It was very low-maintenance. Didn't need to do more, than water the plants once a month.