This question came in through Instagram last week, but it’s one that I hear over and over at my talks and workshops. Though we want to experience and express more joy, in many Western cultures, expressing joy often opens us up to judgment. Being playful or colorful can lead to us being dismissed as childish or unserious, especially in the workplace. And many small joys, particularly those enjoyed by women, like flowers or home decor items, are seen markers of being superficial or self-indulgent.

So how do we ignore the voices that tell us that joy is frivolous and embrace joy anyway?

Focus on why it matters

Close your eyes, and imagine yourself as an 80 year-old. Maybe you’re surrounded by family, with kids and grandkids you dote on. Maybe it’s just you and your partner, or some friends, living somewhere beautiful. You’ve lived a long life, filled with ups and downs. What does that life look like to you if you express your joy? What does it look like if you hold back?

Whenever I do this exercise, I always find that if I am holding back because of fear of judgment, the resulting life looks a lot smaller than the one where I leaned in to my joy. Maybe I did a lot of the same things, but all along the way, there were slivers of life missing. And over time, these really add up.

Recent research has shown that in Western cultures, we’re more likely to put off in-the-moment joy (what the researchers call “experienced happiness”) in favor of things that we think will bring us greater satisfaction later in life (what the researchers call “remembered happiness”). We put off time with our friends so we can work longer hours and get a promotion. Or we tell ourselves that it’s better not to risk embarrassment dancing at a festival, because that’s just a momentary pleasure, while the judgments of others will be with us a long time. The problem is that when imagine ourselves looking back at our lives, the kind of happiness that is really important turns out to be joy: these fleeting experiences that we’re so quick to postpone. In fact, when thinking about their lives as a whole, 79% of people value joy, but when thinking about their next hour, only 49% of people prioritize this.

We forget that our lives are made up of these small moments. The more we are able to remember this, the less willing we’ll be to sacrifice them for other people’s opinions.

Rewrite your own judgments

When I was in my 30s and single, I often found myself feeling self-conscious. I didn’t mind being alone. My idea of a joyful day was biking to the park, sitting under a tree with a good book, maybe meeting a friend for a cup of tea before browsing the shops in my neighborhood and then cooking myself dinner. But as I would sit under my favorite tree with my book, I began to imagine what all the people around me were thinking when they saw me. “Look at her,” I imagined them thinking, “isn’t it pathetic? Can’t find a man so she just has to sit in the park alone with a book. No one wants her. So sad.”

This tape repeated itself just about anywhere I was alone. When I confided in a friend that this thought often occurred to me when I was doing things alone, she pointed out that the entire monologue had occurred in my head. She mentioned that she actually envied me the ability to have an entire free day to sit and read. She gently pointed out that maybe this judgment was more about me than about the people in the park.

So the next time I went to the park I started looking at people and trying to notice what they were actually thinking. I didn’t notice any pitying looks or conspicuous stares. To my relief, most people did not really seem concerned with me at all. But my real revelation came when I saw another woman, like me, in the park with a book. “Poor her,” I started thinking. “Just like me she’s all alone.” And then I stopped myself. I didn’t know her. Maybe she was lonely, but maybe she was perfectly content. The story I was telling myself about her was the same story I imagined others telling about me. I looked at her again and I thought, “Wow, look at her. So free! Not a care in the world.” And then I started to try think about myself the same way. The story didn’t change overnight, but after doing this for awhile, it did start to change. And eventually, it opened me up to doing other things alone, like taking myself out to dinner, or traveling to Iceland by myself.

If there’s a judgment we’re afraid of, often the reason is that it’s a judgment we ourselves hold. The solution is to look at times that we have judged others frivolous, or childish, and rewrite those judgments. When we do, we realize that others don’t care about what we do as much as we thought.

Start small

When I was working as a brand consultant, for many years I used to wear mostly black and grey clothing in an attempt to “look the part.” But I often wore brightly colored shoes. I figured that most of the time, people weren’t looking at my feet, so it was a safe way to experiment with bringing something joyful into my workplace. I was surprised to find that the shoes often became conversation-starters. People didn’t see me as less serious because of the shoes. Actually they made me more approachable, and before long, it stopped feeling like a risk to wear them.

By starting small, you give people time to adjust to changes, and you give them the opportunity to react in ways that might surprise you. Even people who seem likely to judge may find they like the changes in spite of themselves, creating space for bigger changes down the road.

Start in private

When I was in high school, a friend of mine told me not to sing along with the radio because my voice was too “flowery.” I did as I was told, immediately embarrassed of my flowery voice. And even thought I loved singing, I mostly stopped singing around people. I was so ashamed of my voice, I developed a fear of karaoke.

But one day, while I was living in Sydney, I decided I wanted to try to conquer my fear. I picked a song and I started to sing it to myself in my apartment, memorizing the words, feeling out the notes. I found a work friend who was equally afraid of karaoke and one day after a work happy hour we went to a karaoke place and put our songs in the queue. We each stood up there alone and belted out our previously prepared tunes in front of about 30 other karaoke fans. I have no idea if it was good or bad. All I remember is that I felt free.

Starting in private (or in front of strangers!) allows you to get comfortable with a change before you expose yourself to the judgments of others. When we make big changes, often at first they don’t quite feel like a part of our identity. They’re something we’re trying on, and that feels particularly vulnerable. But when we’re at ease with a new style or a new behavior, it makes it more likely that others will see that ease and respond to it.

Find your people

If you find that a lot of people in your life don’t understand what gives you joy, then maybe it’s time to find some new friends. I’m not saying that you need to discard important relationships, but rather that it can be helpful to expand your friend circle to include people who understand your joy and can help you feel safe to express yourself. Attend Meetups or classes to find people with common interests. Fellow nature-lovers are unlikely to roll their eyes when you talk about your latest wild animal sighting. Fellow crafters are going to understand your desire to “put a bird on it.”

Sometimes bringing an old friend into a new dynamic can help them see your joy in a new light. When they see you freely enjoying yourself, it might even free them to let go of their own judgments and find their own joy.