If you’ve changed your eating habits to focus on healthier foods and gotten serious about your workouts, you might expect to start shedding pounds. But the reality is that despite what you’ve been led to believe, weight loss is more complicated than calories in versus calories out. If you’re trying to lose weight, check out these habits that might be interfering with your efforts.

If you typically eat a muffin or avocado toast for breakfast, you might need to boost your protein intake. Research suggests that a protein-rich breakfast can help manage hunger better, which may make you less tempted to supplement with a mid-morning snack.

Protein is important at lunch and dinner, too. If you’re routinely eating salads or sipping on gazpacho without any protein accompaniments — like a boiled egg, yogurt, beans, meat, poultry or fish — it could lead to declining muscle tissue over time, which means your metabolism will start to slow down and make it harder for you to lose weight. Aim for 20 to 35 grams of protein per meal (a portion about the size of a smartphone). But keep in mind that to boost weight loss, you may need to cut down on something else, like the starchy portion of your meal.

When you’re short on sleep, it alters your body’s appetite-regulating hormones, spiking the levels of the hormones that tell you when you’re hungry and reducing levels of the ones that tell you when you’ve had enough to eat. The net-net: You’re hungrier than normal. In one study, women who slept for five hours a night (instead of the recommended seven hours) were 32% more likely to experience a 30-plus pound weight gain during the 16-year study period.

When you’re sleep deprived, the type of food you crave can change, too. Studies suggest that it leads to an increase in activity levels of the reward circuitry in your brain, so donuts and other high-calorie foods are more irresistible. If you aren’t meeting the sleep guidelines, start off by creating a set sleep schedule, setting an alarm to go to bed earlier at night and staying consistent about going to bed and waking at routine times every day.

A mere 5% of adults are participating in exercise for 30 minutes a day, so any exercise deserves props. Physical activity is tied to a lower risk of heart disease and diabetes, and staying active can help you maintain your weight more easily. But if you’re exercising to drop pounds (rather than maintain your weight), you might need to re-think your motives. Time and again, research points to the fact that when people exercise, they’re hungrier and therefore, eat more — enough to make up for the calories they worked off.

Studies also show that when coupled with diet, exercise produces virtually the same weight loss as diet alone. Exercise because it makes you feel better, lifts your spirits and makes you feel stronger, more energetic and empowered. These motivators might even help you lean in toward activities you enjoy rather than ones you think you should be doing, which ups the odds you’ll stick with your fitness routine. You’ll get health benefits from as little as 15 minutes of exercise a day — the benefits drop off after 60 minutes a day, so aim to get somewhere in that range.

A little stress here and there isn’t a big deal, but when it’s persistent, your body responds by continuing to pump out the hormone, cortisol, which leads to an increase in appetite and eating. Stress also impacts your food choices. There’s a reason why you crave comfort food, like pizza, instead of a salad when you’re stressed out.

And here’s the kicker: Studies suggest that when you actively try to lose weight by restricting what you eat, it ups your stress and cortisol levels, which is counterproductive. Exercise, meditation and hugs (be it cuddling a pet or hugging a housemate) are a few good ways to help lower cortisol levels, even when stressful circumstances are beyond your control.

Your body tells you when it’s hungry and when you’ve had enough to eat, but over time, you might have learned to override these signals. When you tune in to what your body tells you, you can respond more appropriately, say, by finding an activity when you’re bored (instead of rummaging through the cabinets for a snack) or by wrapping up a meal when you’re satiated, instead of when you’ve had too much to eat. Developing these listening skills takes practice, but over time, can help you reduce stress eating, minimize overeating and enjoy meals more.

It’s possible to overeat very healthful foods, like avocados, nuts and quinoa, and this can influence your ability to lose weight. These foods can absolutely be part of a weight-loss eating plan, but your portion sizes may need some fine-tuning. For instance, a portion of pasta — even healthier whole grain, lentil and chickpea versions — is 1/2 cup (about the size of a teacup). A serving of nuts is an ounce, or the amount that would fill a mint tin. For nut or seed butter, the serving size is two tablespoons, or the size of a golf ball. An easy hack to help you fill up on proper portions is to reverse your ratio of grains to veggies on your plate, having twice the amount of veggies as grains or pasta. This trick helps downsize grain and pasta portions while keeping your plate full of food.

There’s a strong link between eating processed foods and your weight. One possible, science-backed explanation: You eat faster and don’t fill up as quickly on very processed foods, so you wind up eating more. Just think how much easier and faster it is to wolf down a fast food burger than it is to consume a piece of salmon with a side of broccoli and brown rice. Or compare how quickly you can eat an oat-based granola bar to a bowl of oatmeal.

In one year-long study, dieters were assigned to either a low-fat or low-carb eating meal plan, with both groups losing a similar amount of weight. Despite the different dietary approaches, there were some commonalities about how both groups ate. Both limited sugar and overly processed foods and amped their veggie intake while also eating more whole foods. If you want to simplify your eating habits and lose weight, this is a good way to go.