Hello Techies,
In this article, we are going to discuss how to automate computer using PyAutoGUI.
Doing the same thing over and over again can make you less interested. Automating these types of activities can save your lot of time and help you become more productive in other areas. Then why not try new possibilities which help you doing things using advanced technology.
Many modules can do these things, but in this article, we will use a module called PyAutoGUI to automate GUI and desktop using Python.
PyAutoGUI is an automation module provided by Python to programmatically control keyboard and mouse functions. This module contains almost all the functions that can be performed by keyboard and mouse. We can use these functions to automate mouse and keyboard actions.
Table of Contents
Requirements
- Install Python on your machine. To download Python, click here.
- Any editor, For example, Pycharm or Visual Studio.
What you will get to learn
- Mouse movement and click automaton.
- Keyboard key press automation.
Projects Setup
To start any python-based project we have to follow common steps of installation. So if you have done all the installation then Cheers!!.
If you have not yet done with this installation then you can refer to my previous blogs. The links are below.
For Windows machine: https://techpluslifestyle.com/technology/how-to-install-django-3-on-windows
For Linux machine: https://techpluslifestyle.com/technology/how-to-install-django-3-on-ubuntu/
Note: No need to install any packages using pip just follow the step till env activation.
Before using the functions of the PyAutoGUI module, you need to install it. To install this module, run the following command in the terminal.
pip install pyautogui
Now PyAutoGUI is ready to use.
Mouse movements and clicks automaton
Now we will see, how to control mouse movements, clicks, drags, and scrolls.
PyAutoGUI size() and position() method
Before automating mouse actions, it is good to know the resolution of the display. We can get the resolution of the display by using the .size() method.
Example: How to get the screen resolution using Pyautogui?
import pyautogui print(pyautogui.size())
Output
Size(width=1366, height=768)
Now, let’s check how to get the current coordinates of the mouse. You can use the below code.
import pyautogui print(pyautogui.position())
Output:
Point(x=304, y=619)
By using the position() method of PyAutoGUI, you will get the current position of the cursor.
Now will see how to check if the cursor is on the screen or not. For this, you can use the below code.
Example: How do I find Pyautogui coordinates?
import pyautogui x, y = pyautogui.position() print(pyautogui.onScreen(x, y))
Output:
True
Moving the Mouse on Display
So we know some basic methods of PyAutoGUI and the resolution of our display. Now we can move the mouse anywhere on the display within the limits of the display’s resolution. We can move the mouse across the screen using the .moveTo() method of PYAUTOGUI.
Python script to move the mouse every minute:
import pyautogui pyautogui.moveTo(100,100,duration=1)
We have passed three arguments to the .moveTo() method. The first two arguments are mandatory and indicate the destination location but the third argument is optional. If you want the movement to be immediate, you don’t need to pass the duration as the third argument.
If you want to move the mouse cursor relative to its current position, i.e. you want to move left, right, up, or down from where the mouse is currently, a method to do that is moveRel(), which means move relative.
Example: How do I move the mouse in Pyautogui?
import pyautogui pyautogui.moveTo(100,500,1) pyautogui.moveRel(500,200, 1)
Clicking, Scrolling, and Dragging Mouse
PyAutoGUI also provides some more methods like click, scroll, and drag, where you can automate your mouse with these actions.
A mouse click action can be performed using the .click() method. By default, it provides a left click.
Example: How do you click in Pyautogui?
import pyautogui pyautogui.click() # without arguments pyautogui.click(100,100,1) # with arguments
It has three arguments passed to it, the x, y coordinates, and the duration.
Here, we are telling our mouse to click on the point (100, 100) after one second. So when your mouse moves to the search bar, which we specified by those coordinates, it will click on it.
The mouse has three options to click; Left, middle, and right mouse buttons. So, we have to specify which button to press.
The left mouse button is usually the default. For example, pyautogui.click(100 ,100, button=’right’). This code right-clicks the point (100, 100).
It also provides a double-click action using the pyautogui.doubleClick() method.
Example: How do you double click in Pyautogui?
import pyautogui pyautogui.doubleClick() pyautogui.doubleClick(100,100,1)
PyAutoGUI also provides a scroll action to scroll the mouse up or down. To do this action PyautoGUI provides the .scroll() method.
This method takes an integer as an argument, then scrolls up or down depending on the integer. If you pass a positive integer it will scroll up and for a negative integer it will scroll down.
Example: How to scroll up and down using PyAutoGUI .scroll() method
import pyautogui pyautogui.scroll(50) pyautogui.scroll(-50)
The last method for mouse events is dragging, PyAutoGUI provides dragTo() method for this event.
PyAutoGUI’s dragTo() functions have the same parameters as the moveTo() and move() functions. Additionally, they have a button keyword that can be set to ‘Left’, ‘Middle’, and ‘Right’ for the mouse button to hold down while dragging.
Example: How to scroll up and down using PyAutoGUI .scroll() method
import pyautogui pyautogui.dragTo( 100, 100, 2)
Keyboard key press automation
Pyautogui write() Function
pyautogui.write(‘Python’) is a keyboard method used for typing automation. This function will type the characters in the passed string. To add a delay interval to each letter key press, pass an int or float for the interval keyword argument.
Example: How do you write text in Python?
pyautogui.write('Python world!') # prints out "Python world!" instantly
pyautogui.write('Python world!', interval=0.25) # "Python world!" prints with a quarter second delay after each characterYou cannot press the Shift or F1 key, you can only press a single letter key with write().
PyautoGUI press(), keyDown(), and keyUp() Functions
Now let’s check Pyautogui’s keyboard press() method.
To press any keyboard key, call the press() function and pass it a string from pyautogui.KEYBOARD_KEYS such as enter, esc, f1.
Example: How do you press the key in Pyautogui?
pyautogui.press('enter') # press the Enter key
pyautogui.press('f1') # press the F1 keyTo press multiple keys, just like write(), pass a list of strings to press(). For example:
pyautogui.press(['left', 'left', 'left', 'left'])
Or you can set left presses:
pyautogui.press('left', presses=4)The following are valid keyboard keys:
['\t', '\n', '\r', ' ', '!', '"', '#', '$', '%', '&', "'", '(',
')', '*', '+', ',', '-', '.', '/', '0', '1', '2', '3', '4', '5', '6', '7',
'8', '9', ':', ';', '<', '=', '>', '?', '@', '[', '\\', ']', '^', '_', '`',
'a', 'b', 'c', 'd', 'e','f', 'g', 'h', 'i', 'j', 'k', 'l', 'm', 'n', 'o',
'p', 'q', 'r', 's', 't', 'u', 'v', 'w', 'x', 'y', 'z', '{', '|', '}', '~',
'accept', 'add', 'alt', 'altleft', 'altright', 'apps', 'backspace',
'browserback', 'browserfavorites', 'browserforward', 'browserhome',
'browserrefresh', 'browsersearch', 'browserstop', 'capslock', 'clear',
'convert', 'ctrl', 'ctrlleft', 'ctrlright', 'decimal', 'del', 'delete',
'divide', 'down', 'end', 'enter', 'esc', 'escape', 'execute', 'f1', 'f10',
'f11', 'f12', 'f13', 'f14', 'f15', 'f16', 'f17', 'f18', 'f19', 'f2', 'f20',
'f21', 'f22', 'f23', 'f24', 'f3', 'f4', 'f5', 'f6', 'f7', 'f8', 'f9',
'final', 'fn', 'hanguel', 'hangul', 'hanja', 'help', 'home', 'insert', 'junja',
'kana', 'kanji', 'launchapp1', 'launchapp2', 'launchmail',
'launchmediaselect', 'left', 'modechange', 'multiply', 'nexttrack',
'nonconvert', 'num0', 'num1', 'num2', 'num3', 'num4', 'num5', 'num6',
'num7', 'num8', 'num9', 'numlock', 'pagedown', 'pageup', 'pause', 'pgdn',
'pgup', 'playpause', 'prevtrack', 'print', 'printscreen', 'prntscrn',
'prtsc', 'prtscr', 'return', 'right', 'scrolllock', 'select', 'separator',
'shift', 'shiftleft', 'shiftright', 'sleep', 'space', 'stop', 'subtract', 'tab',
'up', 'volumedown', 'volumemute', 'volumeup', 'win', 'winleft', 'winright', 'yen',
'command', 'option', 'optionleft', 'optionright']Let’s check keyUp(), and keyDown() method.
Example:
pyautogui.keyDown('shift') # hold down the shift key
pyautogui.press('left') # press the left arrow key
pyautogui.press('left') # press the left arrow key
pyautogui.keyUp('shift') # release the shift keyPyAutoGUI hold() function
To make key holding convenient, the hold() function can be used as a context manager and passed a string from pyautogui.KEYBOARD_KEYS such as shift, ctrl, alt, and this key will be held for the duration with the context block.
with pyautogui.hold('shift'):
pyautogui.press(['left', 'left', 'left', 'left'])which is equivalent to:
pyautogui.keyDown('shift') # hold down the shift key
pyautogui.press('left') # press the left arrow key
pyautogui.press('left') # press the left arrow key
pyautogui.press('left') # press the left arrow key
pyautogui.press('left') # press the left arrow key
pyautogui.keyUp('shift') # release the shift keyPyautogui hotkey() Function
Pressing and holding the ‘Alt’ key and then pressing the ‘F4’ key and releasing the ‘Alt’ key after releasing the ‘F4’ key is very complicated. This can only be done using the .hotkey() method. The following example will illustrate the concept.
Example:
pyautogui.hotkey('ctrl', 'shift', 'esc')Conclusion
The code above demonstrates simple mouse and keyboard automation using the PyAutoGUI library.
You can modify the PyAutoGUI above code as per your requirement and use it to automate your daily repetitive tasks.
I hope you understand the concept of the Python PyAutoGUI . If you have any doubt please comment down below.
