Sure. Just like arguing that if someone only owned a single b&w tele and second-hand pants, they must have been "pretty poor" is a completely false assumption.
So arguing that if someone wasn't on food stamps, they must not have been "pretty poor" is a completely false assumption.
But if someone does those things because they can't afford a color TV, or new pants, then it is a decent way to determine whether someone was "pretty poor".
If we can't define levels of poverty based on what sorts of things people can afford to obtain, then we don't really have any meaningful standard at all. While I'm sure that some middle class kids shopped at the goodwill store to buy cheap stuff out of a sense of frugality or something, for my family it was a necessity. We could not afford brand new clothes. Period. We were able to afford the most important basics, but had to scrimp and save wherever we could in order to make ends meet.
If that doesn't qualify as "pretty poor", then what does? I see it as being just above poverty. You're not desperate. You're capable of getting by without help. But just barely. What other definition would you suggest for "pretty poor"?