Why did the Buddha insist on total abstinence from sexual expression for his monks undergoing the gradual training?
This is actually a good question, but to answer it thoroughly will require unpacking Buddha dhamma (philosophy) from Buddha adhamma (dogma).
1) The Sanskrit term 'sangha' is not the property of Buddhism. It existed in Aryan philosophy long before Siddhartha Gautama was born, and he knew that.
2) The Sanskrit term 'sangha' does not mean 'priesthood' or 'monks,' it means 'community,' in the same way 'church' does not mean a place but more accurately means "those who embrace Christianity."
3) Siddhartha Gautama had both lay and monastic followers (sangha).
4) The Pali canon states in numerous places that Siddhartha Gautama had both lay and monastic followers (sangha), who belonged to one of the 4 levels of noble beings (arya pugala), and most notably he had lay followers who he recognized as arahats.
5) Therefore, celibacy was not a requirement for all of Siddhartha Gautama's followers (sangha), nor for the attainment of arahatship.
6) However, celibacy, among other disciplines, was part of the monastic code (vinaya) of the Buddhist sangha.
Now, to answer why celibacy was a requirement for the monastic code (vinaya) of the Buddhist sangha we have to look at the bigger picture.
1) When one becomes an arahat, which requires 4th jhana mastery, one loses interest in sex.
2) The entire trajectory of the Noble Eightfold Path is to answer the Four Noble Truths, which is about eliminating suffering (dukkha), which requires the attainment of arahatship, which requires 4th jhana mastery.
3) A married person who has no sex-drive ceases to be a useful marriage partner.
4) Additionally, the attainment of arahatship, which requires 4th jhana mastery, in most cases, is a full-time job, which means there is no time for a relationship.
5) Therefore, celibacy is skillful means.