You will notice that Primary Campaign Source field in Opportunity is always the same with selected Primary Campaign Source in Campaign Influence. But, only one campaign can tick as Primary Campaign Source in Opportunity.
From above screenshot, you notice two campaigns related in Opportunity Opty Hello, but My One is the one marked as Primary Campaign Source.
Now, if we go to Campaign and look for both campaigns above. There is Opportunity related list in the Campaign. But only in campaign selected as Primary Campaign Source (for this case My One) , the Opportunity will be displayed. For the other campaign You Two, although it is listed in the Opportunity page layout, it will be not shown in Campaign, because it is not Primary Campaign Source.
This is out of the box Salesforce functionality and no way to modify it, unless you totally do not want to use Campaign Influence & Primary Campaign Source, you can create junction object to relate Campaign and Opportunity.
Reference: What is Campaign Influence?