Cane toads can kill humans. The toxin is an irritant on the skin. If ingested or comes in contact with the blood it can kill you in high enough concentrations.
Small concentrations <5mg (1mg=0.001g or 1/1000 gram). This is very small toxin levels of Bufotenin (which is found in cane toad excretions). The above is in humans.
It affects respiratory, which is a tightening of the throat and a heavy chest. Increased amounts cause the throat to block and the intercostal and diaphragm to rapidly contract. This means that you can't even take in small breaths, as the muscles are locked. It causes death by asphixiation.
A number of organisms consume cane toads, and other toads with the toxin. These are mostly region specific predators, which are adapted to deal with the toxin.
So yeah, it can kill fast. Depending on the amount, and concentration taken in. So yeah, not the best things to eat.