I'd say from first hand experience it's possible to think without language. I used to be a linguistic thinker where I predominantly thought in words/phrases, and have "dialectics" going on in my head. You know that "inner monologue" of; "Oh I should've said that, This makes sense, or I'll say this when I get a chance". That is what I'm going to assume a lot of people take into consideration in a person says they're "thinking". Provided I'm not too crazy.
To me I think it may be a stepping stone to "higher thinking" (loose terminology/definition as to what's higher). Much like the older forms of philosophy, and continuing on to the present where debates between thoughts, and arguments are played out in "external world" with other people. I'll say something, and you say something in response. That is considered "normal dialectics", but if a person is capable of having an "inner monologue" where they can, "reconsider, or consider things prior" like mentioned above. Who's to say that they can't turn their "inner monologue" into an "inner dialogue"? This is where I think a person develops mentally (opinion, and assumption). You know adopt a "stage voice" for one of their "monologue" tones? Like say the secondary voice that says, "Oh you could've done this better.", that inner critic is actually there.
I've once read somewhere, and witnessed firsthand that when it comes to reading things visually. There are those who read aloud to understand the passage, and then overtime it came to be realized that you didn't have to read aloud to read. Thus it came to a point where as a reader developed in readership skills, they often started reading with a "silent mutter". Watch yourself, or another as they read, and you may see it. They're silently mouthing the passages as they read them.
Drawing an analogy to thinking, I would say that when we're thinking to "ourselves" or so we "think". There may be a chance that we're doing sub-vocal mutterings to ourselves of what our ideas are. We're "talking" to ourselves, in other words. And once you start monitoring, and develop that idea that there can be "multiple people" inside your head. Not in an identity sense but as vectors to portray them as. Mental concepts if one will say.
Like "talking to a imaginary rock" inside your head, while picturing the classic repose of Hamlet while eulogizing Yorrick. ("Alas poor Yorrick," *while holding his chapless skull* , "I knew him well...").
Overtime now, I've turned/developed into a "visual thinker" where I can imagine multiple dimensions in my "mind's eye". As a challenge, I can ask you to visualize a 3x3 cube (a Rubix cube essentially). If you think you can, try imagining a cube upon a cube in multiple ways. A tower of six smaller cubes that is three small cubes deep, and three small cubes wide. If they are stacked vertically, or do a rotation with it in your mind. It just occurred to me, that I probably could "play" Rubix cube in my mind now..., but that's something for if I really want to "tune out" if I need to, I guess.
As a final point, I'd simply say, that there are no limits to the conventions of "thought". It is all dependent upon the "ontological bases" that one has accepted, and/or endorsed (the "boxes" they have, or they think and categorize by).