There are many species of horned worms and yes, some are toxic depending on what they've been eating. I wouldn't feed using wild ones. You must be careful about feeding any wild larvae (caterpillars).

We have an abundant soft-bodied caterpillar here that was introduced to control tansy ragwort - a toxic weed. Unfortunately, the local northwestern alligator lizards love to eat these caterpillars and as a result the lizard population has been decimated where these caterpillars (and tansy) are. The caterpillars retain toxins from the weed and poison the lizards. Birds seem to be smart enough to leave them alone.