Basically the idea is socialism would be implemented by smaller, local organizations with either no government or as limited a government as practically possible, depending on the exact strain of thought.

There are actually many different schools of socialist thought. Most people have only heard of a few because of US and Soviet propaganda. In fact many libertarian socialists consider countries like the USSR and China to be “state capitalist” countries that are only pretending to be socialist for propaganda purposes. Similar to how the US markets itself as a paradise for ordinary people while also subjecting them to various forms of oppression and poverty.