Lots of stuff in Houston to do at all hours, and HUGE diversity of cultures (esp. Asian on West side) unavailable in other TX cities, if either of those is important or interesting to you. Also close to the coast, Piney Woods, 3 hours from Austin and San Antonio and hill country roads, 2 hours from trackdays at TWS....
My wife and I grew up there but moved to San Antonio 22 years ago because we LOVE SA and as we got older got tired of Houston's obvious drawbacks (Flat. Humid. Crowded. Traffic. Huge.)
Don't worry too much about what you hear from people in other TX cities. People in SA throw down on Houston because they tend to be intimidated by its size and complexity. A lot of people in Dallas have an issue with being 2nd biggest so they'll always diss Houston to boost their own self-esteem -- you know the type, coolly condescending but plainly evident. Austin? Well, no place is as good as that if you were to ask them. And they might be right.
I suggest you take or don't take the job on its own merits. Also check on where the job is. Houston sprawls all over the place and with funky/inconsistent zoning one part of town can be vastly different from others. For example, the Woodlands area is easy to live in and nice looking and you'd never know you were in the midst of big city chaos, kind of like Central Park or Prospect Park in NYC.
If you take the job and go there but find it a bad fit, split!
Good luck either way!