What is a Marking Menu?
Defining our need
The investigation

The premises
We found the culprit!

Some code

Hello, I'm a professional hacker
Advanced tweaking
Sub menus
Contextualization
Connecting to a scriptNode

What is a Marking Menu?

Good question, a marking menu is a contextual radial menu which pops up when you right click with your mouse in Maya's viewport:

Composed with a maximum of 8 menuItem which can have sub menus, their positions are called the radial position, named after the four cardinal points

Defining our need

Clearly, speaking of tweaking, Maya allows us to go really far, except a few static properties, alors everything can be changed. However, one element is still static, that's the marking menu, you can edit few settings by going in the preference, but that remains very simple, which depends on the selection. This tool literally saves you time when you know how to use it properly.

So we're gonna try (sucessfully =)) in this tutorial to change this static property, to be able to edit and modify it with our needs, for instance ;

a rigger : changing the marking menu to allow hom to directly create constraints, if the selection is a joint, he can directly creates an IK
an animator : allowing the animator to switch between IK / FK the selected controller, if the attribute exists, and reset the transforms in translation, rotation or both
during the render process : create an exclusion light set in one click, dupe a shader of the selected object
etc...

Let's try to find how Maya fills this menu and how we can change it...

The investigation

The premises

Obviously, the first tryouts on these researchs where far more laborious... So we'll avoid this painful part of the story and directly go to the interesting discoveries =) !

Alright, if we open our verbose friend the scriptEditor, activating the option History→Echo All Commands. Using the right click of the mouse on an empty space (everybody knows the space isn't empty, but whatever !) we should have a line in the Maya's output looking like that ;

  buildObjectMenuItemsNow "MayaWindow|(...)|modelPanel4|modelPanel4ObjectPop";

Let's create a circle and try again ;

  buildObjectMenuItemsNow "MayaWindow|(...)|modelPanel4|modelPanel4ObjectPop";
  dagMenuProc("MayaWindow|(...)|modelPanel4|modelPanel4ObjectPop", "nurbsCircle1");

Well well... That's interesting, the contextual display of the menu calls a second function, which is the dagMenuProc procedure, let's try to find it using our second curious friend "Notepad ++", searching the Maya's folder the call of this proc, you'll find a file containing this word a lot of time, namely the dagMenuProc.mel file, can't be easier =). You'll find this file in %MAYA_FOLDER%/scripts/others/dagMenuProc.mel, let's open it... Aww, this is a big file !

We found the culprit!

Now that we found the correct file, let's have a walk inside if we can find some 'main' function of this whole dagMenuProc... Let's say... The biggest one, namely createSelectMenuItems. This is the goal of our quest, the function called by Maya when the mouse's right button is pressed, we also note that this functions expects two arguments$parent and $item, which are two arguments sent by Maya in the piece of code quoted above.

A glimpse on the function's content tells us interesting things, even if MEL language isn't the shortest language of the world, we can summarize the function this should be something like that ;

proc createSelectMenuItems($parent, $item)
- Declaring the variables
- We look at the type first argument $item and we define the corresponding variable to True
- Condition on the type of $item
We fill our marking menu with menuItems for each needed positions
- We parent our new menu to $parent

Some code

Hello, I'm a professional hacker

Alright, now that we know which is the function called by Maya, we're simple going to redeclare it to Maya, lets open our scriptEditor et create a new MEL tab, we're going to start with a nasty test ; redeclaring an empty function :

  global proc createSelectMenuItems(string \$parent, string \$item){}

Now maybe we should try something better... For instance, adding a menu on the top position (N). If we look at the original function, we see that a menuItem is declared this way ;

  menuItem -label $enableIkHandle
    -annotation (uiRes("m_dagMenuProc.kEnableIKHandleAnnot"))
    -echoCommand true
    -c ("ikHandle -e -eh " + $handle)
    -rp $radialPosition[4];

We also note that at the end, the function setParent -menu $parent; is called, which will parent our newly created menu to the Maya's UI.

Let's try to redeclare createSelectMenuItems as follows ;

    global proc createSelectMenuItems(string $parent, string $item){
      menuItem -label "test" -radialPosition "N";
      setParent -menu $parent;
    }

Ok good ! We see our first menu popping :) !

Thank to the -command argument - or simply -c we will be able to add a command to our function, which will be executed everytime the menu is selected:

  global proc createSelectMenuItems(string $parent, string $item){
      menuItem -label "test" -radialPosition "N"
          -c "print \"Hello world\"";
      setParent -menu $parent;
  }

Generally, if not always, in programming language the use of antislash \ allows you to 'pass' a character, which means it won't be interpreted by the programming language, in the previous example, the use of \" allowed us to insert inside the first string a second string with double quotes. The first one is interpreted by the MEL interpreter, and the second will be interpreted when the command is executed.

Advanced tweaking

Actually, seeing the menuItem documentation, we can go deeper that what Maya shows us, for instance, did you ever see a marking menu in italic ? Quite simple !

  global proc xxx(string $parent, string $item){
    menuItem -label "test" -radialPosition "N"
      -c "print \"Hello world\"";
      menuItem -label "italic menu" -radialPosition "E"
      -c "delete `ls -sl`"
      -itl ;
    setParent -menu $parent;
  }

Ready to go =)

Sub menus

Implementing a sub menu is also quite simple, we just need to set the flag subMenu then parent our sub menus to the parent menu (the one on which we set the subMenu flag). A simple example would be ;

  global proc createSelectMenuItems(string $parent , string $item ){
      menuItem -label "test" -radialPosition "N"
          -c "print \"Hello world\"";
      menuItem
          -label "main menu" -radialPosition "W"
          -subMenu 1;
      menuItem
          -label "First sub menu" -radialPosition "N"
          -command "print \"First\"";
        menuItem
            -label "Second sub menu" -radialPosition "W"
            -command "warning \"Second\"";
        setParent -menu ..;     
    setParent -menu $parent ;
}

The function setParent -menu .. will parent our menus to the first matching result in the creation hierarchy... Which is our 'main menu' menuItem")

And we get:

Contextualization

Well, our system seems to work. Nonetheless this method overrides all the marking menus regardless of the type of the object, to change only the marking menu for one object type, we need to keep the declaration of the variables such as $isBezierObject at the beginning of the MEL function, as well as the condition setting the True status of the correct var:

      
  if (1 <= size($maskList)) {
      $isLatticeObject = ($maskList[0] == "latticePoint");
      $isJointObject = ($maskList[0] == "joint");
      $isHikEffector = ($maskList[0] == "hikEffector");
      $isIkHandleObject = ($maskList[0] == "ikHandle");
      $isParticleObject = ($maskList[0] == "particle");
      $isSpringObject = ($maskList[0] == "springComponent");
      $isSubdivObject = ($maskList[0] == "subdivMeshPoint");
      $isLocatorObject = ($maskList[0] == "locator");
      $isMotionTrail = ($maskList[0] == "motionTrail");
  }
  if (2 <= size($maskList)) {
      $isBezierObject = ($maskList[1] == "bezierAnchor");
      $isNurbObject = ($maskList[1] == "controlVertex");
      $isPolyObject = ($maskList[1] == "vertex");
  }

And only edit the part we need to be modified in our MEL function. Which can be arduous...

Fortunately for you, here we are, with a ready solution to remedy this hardship. So you don't have to edit yourselves this long function, just download this -quite- simple script radialDesigner and run it (see the download section)

The use could not be simpler, you just need to select in the list at top on which kind of object you want to apply the marking menu, then click on the menuItems to edit their codes, appearance, everything, whatever, the images speak by themselves, so just have a look on this demo video =) ;

Connecting to a scriptNode

The second 'tip' of this whole story, is to put our complete redeclaration (of the createSelectMenuItems function) inside a scriptNode which will be loaded each time scene is started, which can be useful in a studio for instance, when a rigger has defined properly this menu he can integrates it in his rig scene, so every animator opening the scene will have the menu override, so they can animate better and faster... maybe stronger =).

The method is quite simple, given what we saw previously, we just need to create a multi-line MEL variable and put all the function inside, then we call this variable with a evalDeferred which will execute the function when Maya is fully loaded, thus after the UI with the default marking menu is loaded.

For the same reasons as before I invite you to use the radialDesigner to achieve that, it will spare you a looot of time, and a large amount of hair pulled to correctly imbricate a string into a string which is itself imbricated into a string with the antislashs =) !

Table of Contents