The distance between two numbers is the absolute value of their difference. Note that the absolute value is necessary in order to make sure that the distance between A and B is the same as the distance between B and A.
So, you have
[tex] d(-4,1) = d(1,-4) = |-4-1| = |1-(-4)| = |-5| = |5| = 5 [/tex]