Ward brook lean to

Heading in this weekend to hike Seymour. I know its about 5.5 miles in to the lean to. I am planning on about 2 hours to reach the lean to, does this seem about right given the trail and 4 guys with full packs?