Writing them in a more abstract languages has advantages - like automatic differentiation.
But they are still tricky code, because if you get them wrong you will find that you spent 500 GPU-years turning random numbers that cause the model to output gibberish into other random numbers that cause the model to output different yet semantically identical gibberish. ML algorithms are, at their core, not particularly complicated code. The question is whether anyone finds it worth doing.
