Go Memory Model Deep Dive: What Every Go Developer Must Know

Lost Orders: a short story of how it all started

In one of our services, we had a background process that collected results from workers and periodically flushed them. In tests — perfect. On staging — fine. In production — every few hours we discovered that some processed tasks simply disappeared.

No errors. No warnings. Just silence — and holes in the data.

The root cause turned out to be painfully simple: a regular counter was being updated concurrently without synchronization. Sometimes operations overtook each other, sometimes increments were lost. Under real load, this turned into financial discrepancies.

The fix was a single line (atomic.AddInt64), but the lesson was far more important: without a happens-before relationship between goroutines, nothing behaves the way you intuitively expect.

1. Why understanding the memory model matters

Almost every serious bug in concurrent code boils down to one question:

"I wrote the value. Why didn't the other goroutine see it?"

The answer is almost always the same: the data was written, but not published.

Meaning:

• you changed something,
• but the compiler or CPU reordered operations,
• and the other goroutine saw stale data.

Go's memory model is strict: without happens-before, visibility is not guaranteed.

This is invisible on small projects. Under load, it becomes a real problem.

2. A small real example

Multiple workers wrote results into a shared structure. Another goroutine periodically flushed those results. A counter tracked how many jobs had been processed.

type Results struct {
    items []Item
    count int64  // problem here
}

func (r *Results) Add(item Item) {
    r.items = append(r.items, item)
    r.count++  // read + write, not atomic
}

No synchronization.

On small loads — fine. Under production load — missing data.

Why? Because count++ isn't atomic — it's three operations:

1. Read current value
2. Add 1
3. Write back

Between those operations, another goroutine could sneak in and do the same. Both read the same value, both increment, both write back — one increment is lost.

The fix:

func (r *Results) Add(item Item) {
    r.mu.Lock()
    r.items = append(r.items, item)
    r.count++
    r.mu.Unlock()
}

Or if you only need the counter:

atomic.AddInt64(&r.count, 1)

Invisible in tests. Deadly under load.

3. Three tools that make your code predictable

Go gives a few simple primitives that guarantee ordering.

3.1 Channels

Send happens-before receive.

ch := make(chan int)
var data string

go func() {
    data = "hello"
    ch <- 1
}()

<-ch
fmt.Println(data) // always "hello"

Channels are one of the safest ways to publish data.

3.2 Mutexes

Unlock in one goroutine happens-before Lock in another.

Reliable, simple, rarely slow.

3.3 Atomics

Atomics define strict memory barriers: what comes before cannot be moved after, and vice versa.

They don't "protect" data — but they guarantee visibility.

4. A single example that explains the model

The classic broken pattern:

type Message struct {
    data  string
    ready uint32
}

func (m *Message) unsafePublish(data string) {
    m.data = data
    m.ready = 1 // can be reordered
}

The CPU is allowed to reorder these operations. Why?

The processor sees: data and ready are different memory addresses. The operations are independent. For performance, it can reorder them.

Result: another goroutine reads ready == 1, but data is still stale.

Correct version:

func (m *Message) safePublish(data string) {
    m.data = data
    atomic.StoreUint32(&m.ready, 1)
}

func (m *Message) safeRead() string {
    if atomic.LoadUint32(&m.ready) == 1 {
        return m.data
    }
    return ""
}

atomic.StoreUint32 creates a memory barrier. Everything before it stays before it. The other goroutine sees consistent state.

One line makes the difference.

5. A minimal lock-free example

Lock-free structures look magical, but they only rely on the same memory model rules.

type node struct {
    next unsafe.Pointer
    val  int
}

type Stack struct {
    head unsafe.Pointer
}

func (s *Stack) Push(v int) {
    n := &node{val: v}
    for {
        head := atomic.LoadPointer(&s.head)
        n.next = head
        if atomic.CompareAndSwapPointer(&s.head, head, unsafe.Pointer(n)) {
            return
        }
    }
}

func (s *Stack) Pop() (int, bool) {
    for {
        head := atomic.LoadPointer(&s.head)
        if head == nil {
            return 0, false
        }
        n := (*node)(head)
        if atomic.CompareAndSwapPointer(&s.head, head, n.next) {
            return n.val, true
        }
    }
}

This works because Go guarantees ordering around atomic operations. CAS creates a full memory barrier.

6. A practical pattern you need in almost any service

A worker pool with proper shutdown.

type Pool struct {
    jobs chan int
    quit chan struct{}
}

func (p *Pool) worker() {
    for {
        select {
        case j := <-p.jobs:
            _ = j
        case <-p.quit:
            return
        }
    }
}

Channels guarantee publication of values and clean shutdown semantics.

7. A tiny test every project should have

Race detector is your friend.

func TestRace(t *testing.T) {
    var x int
    done := make(chan bool)

    go func() {
        x++
        done <- true
    }()
    go func() {
        x++
        done <- true
    }()

    <-done
    <-done
}

Run:

go test -race

The race detector instruments your code and tracks all memory accesses. It will catch data races that only appear under specific timing conditions.

If you don't write such tests — you will find the races in production.

8. The main takeaway

The Go memory model boils down to one simple rule:

If two goroutines access the same memory concurrently and there's no synchronization, Go guarantees nothing.

This isn't a compiler flaw. It's how every fast system works.

Synchronization isn't overhead. It's control.

9. If you want to dig deeper

The official Go memory model documentation is here: https://go.dev/ref/mem

For practical examples, including lock-free structures, race tests, and advanced patterns, check out the Go source code itself — particularly sync/atomic and sync package tests.