a. [tex]y=\cos kt\implies y''=-k^2\cos kt[/tex]
Substituting these into the ODE gives
[tex]-4k^2\cos kt=-25\cos kt\impliesk^2=\dfrac{25}4\implies k=\pm\dfrac52[/tex]
b. [tex]y=A\sin kt+B\cos kt\implies y''=-k^2A\sin kt-k^2B\cos kt[/tex]
If [tex]k=\dfrac52[/tex], then
[tex]4\left(-\dfrac{25}4A\sin\dfrac52t-\dfrac{25}4B\cos\dfrac52t\right)=-25\left(\sin\dfrac52t+\cos\dfrac52t\right)[/tex]
as required, and the same can be said for [tex]k=-\dfrac52[/tex]. (Note that [tex]\sin(-x)=-\sin x[/tex] and [tex]\cos(-x)=\cos x[/tex].)