De La Espriella, 48, won June's run-off on promises to crack down on armed groups, shrink the state by as much as 40 per cent and revive the oil and gas sector.
An ally of US President Donald Trump, De La Espriella represented well-known right-wing paramilitaries during his time as a lawyer.
He has never previously held elected office.
"I have come to close a long chapter of national resignation and, together with the people, embark on the most profound transformation of our destiny," De La Espriella said at the Pichincha Battalion military base in Cali after being sworn in at a nearby university auditorium.
"I send a firm message to the Colombian people: the time has come to restore order, authority, and freedom.
"Members of criminal gangs and narco-terrorist groups have two choices: submit to the rule of law or face the determined response of the Colombian state and its security forces," he told assembled troops at the base during a 75-minute speech, adding the option of peace talks with armed groups had been "completely exhausted".
De La Espriella said the time had come to "definitively eradicate the scourge of illicit crops", adding he would use aerial spraying of herbicides that did not harm human health or the environment to destroy coca.
In one of his first acts as president, Colombia would join the Shield of the Americas program founded by Trump, De La Espriella said.
He also pledged to freeze public spending and pursue tax reform to simplify the country's payment system and boost investment and employment, while preserving popular social programs, some put in place by his predecessor.
"I want to tell Colombia's business leaders, large, medium-sized and small, that this government will be your partner in investing in Colombia," De La Espriella said.
"Once again, it will be a safe and profitable decision for you."
