(1)根据题意得AN=AM-MN=6-x,∵四边形DEFG是矩形,∴DG∥BC,∴△ADG∽△ABC,∴DG:BC=AN:AM,即y:12=(x-6):6,∴y=2x-12(0<x<6);(2)∵当DG=GF时,四边形DEFG是正方形,∴y=x,即2x-12=x,∴x=4,∴y=4,∵△ADG∽△ABC,∴S△ADG:S△ABC=DG2:BC2=16:144=1:9.