If x is the side of the squares being cut out of the sheet, then when the sides are bent up, the box that results has dimensions x, 10 - 2x, and 50 - 2x. The volume is
[tex]V=x(10-2x)(50-2x)=x(500-120x+4x^3)=4x^3-120x^2+500x[/tex]
The derivative is [tex]V'=12x^2-240x+500[/tex]
Set that equal to 0 and solve for x (answers are weird!).
[tex]x=\frac{5(6\pm\sqrt{21})}{3}[/tex]
Use a calculator to plug those into the volume function, then pick the largest.