Let

w = width of the rectangular pen

l = length of the rectangular pen

she has 170 feet of fencing and will use the barn as one side of the enclosure

l + 2w = 170 (we will assume the barn is along one of the the lengths)

l = 170 - 2w

the area of the rectangular pen is:

A = l*w

plug l = 170 - 2w into the last equation:

A = (170 - 2w)w

A = -2w^2 + 170w

Looking at the coefficient for the w^2 term, we see that it is negative. This indicates that the parabola opens downward and finding the vertex will give you the "maximum".

A max = c - (b^2)/(4a), where a = -2, b = 170, c = 0

A max = 0 - (170^2)/(4*(-2))

A max = 3612,5 ft^2

the max area that can be enclosed is 3612,5 ft^2.