The team at xAi have been cooking, rolling out a new feature that let’s Grok watch while you talk to it.
LLMs are rapidly evolving from a chat interface to being more multi-modal and voice is certainly a great way to interact with them. One issue that often comes up is the conversation you’re having lacks the context of what you’re looking at.
In the context of our time on computers, it’d be awfully nice to be able to share that context with Grok as you talk to it, to get context-specific responses.
An example of this could be programming, where you’re working on a specific piece of code and want to avoid the copy+paste cycle, instead you could simply share your IDE window and ask Grok about the error or issue you’re fighting.
You could also have it review your website and using Grok Voice Screen Share, scroll the website, browse through different pages and get feedback on it, live, talking through it as if you have a consultant 24×7 you can lean on.
The only real boundary to this is how comfortable you are sharing content with Grok. You could be trying to architect a budget in Excel, or a new product design in photoshop, even get help playing a game, the possibilities are endless.
While other LLMs offer voice mode and the capacity to share your screen, these are isolated to the mobile apps. This is great, but the majority of my week is spent at the desktop, being productive, and that’s where this feature can have the biggest benefits.
It’s not clear when this launched exactly as it’s not even listed on the official https://x.ai/voice page yet, but it is confirmed by Elon Musk that the feature is new.
If you’ve used Grok Voice Screen Sharing, or have examples you may like to try with it, leave them in the comments.
