Answer:
[tex]2xy^2(xy-1)(xy-3)[/tex]
====================
Given expression
[tex]2x^3y^4-8x^2y^3+6xy^2[/tex]
The greatest common factor of all three terms is [tex]2xy^2[/tex].
Factor this out:
[tex]2xy^2(x^2y^2-4xy+3)[/tex]
Complete the square:
[tex]2xy^2(x^2y^2-4xy+4-1)=[/tex]
[tex]2xy^2((xy-2)^2-1)[/tex]
Factorize further using the identity for the difference of squares:
[tex]2xy^2(xy-2+1)(xy-2-1)=[/tex]
[tex]2xy^2(xy-1)(xy-3)[/tex]