kindred/create
1
1
Fork 0
Code Issues 53 Pull Requests Actions Packages Projects 9 Releases 2 Wiki Activity

Made the interval that the Python profiler runs at configurable

Browse Source 
Added a "User parameter:BaseApp/Preferences/PythonConsole/ProfilerInterval" int
property which sets how often (in milliseconds) the Python profiler runs while
Python code is running. Setting this value to zero will totally disable it.

Also added a preference in the Python console screen which allows the user to
set the value of this property to between 0 (disabled) and 5000 (once every 5
seconds). (+1 squashed commits)

Squashed commits:

[cca88ac633] Made the Python profiler only run when the console is running code

This has two purposes. First, it prevents a performance impact from running the
profiler whenever Python code is running. Second, it prevents crashes caused by
Qt's process events function being called too frequently. When the Python code
is running in the console, it monopolizes the main thread and prevents events
from being processed. Therefore, causing events to be processed in the callback
should not force events to be processed too frequently, because the normal loop
is being prevented by the Python code. (+1 squashed commits)

Squashed commits:

[45f86917e6] Made long-running Python code not freeze the GUI without multithreading

Removed the background thread running Python code and replaced it with a custom
profiler which the Python interpreter runs frequently (at every Python opcode I
believe) on the main thread whenever Python code is running. The profiler will
make Qt process any new events every 200 ms, preventing "App not responding"
messages and making sure any Ctrl+C keypresses will be processed. This prevents
the previous issue where running anything GUI-related from Python would crash
the program (because Qt isn't thread-safe). (+1 squashed commits)

Squashed commits:

[0ef7d810b3] Made the process of getting thread IDs cross-platform compatible

Instead of using <threads.h>, now the standard <thread> header from C++
11 is used to find the thread ID, since <threads.h> is apparently not
available on Windows. (+1 squashed commits)

Squashed commits:

[74c7b867f2] # This is a combination of 2 commits.

Python from the console now runs in the background

In a nutshell, all Python code which is input from the interactive console
now runs in a seperate QThread which runs in the background, instead of
on the UI thread. This means that long operates operations will no longer
cause the app to display an "App not responding" message, because the UI
thread is now free to keep running unencumbered.

However, it is still not possible to run multiple Python statements at
the same time. If the user tries to run some Python while a previous
statement is still being processed, instead it will display an error
message in the console stating that the previous command is still
being processed.

I also added a seperate QTimer which runs once every 100ms to flush any
output from the Python code back to the console. I can't flush the output
to the console from the background thread because the relevant Qt5 code
is not thread-safe (it causes random segfaults). So I added this timer
as a work-around, since it runs in the main UI thread.

Implemented Ctrl+C keyboard interrupts in the console

This is implemented by detecting a Ctrl+C key event in the Python
console in the main Qt UI thread, and sending a keyboard interrupt
to the background thread that runs the Python code.
This commit is contained in:
Michael Tuma
2022-04-06 17:45:08 -04:00
committed by wwmayer
parent f1ef5c4729
commit 32c5968069
7 changed files with 375 additions and 7 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
src/Gui/CMakeLists.txt
View File

@@ -666,6 +666,7 @@ SET(Editor_CPP_SRCS
PythonConsole.cpp
PythonConsolePy.cpp
PythonDebugger.cpp
PythonTracing.cpp
PythonEditor.cpp
SyntaxHighlighter.cpp
TextEdit.cpp
@@ -677,6 +678,7 @@ SET(Editor_HPP_SRCS
PythonConsole.h
PythonConsolePy.h
PythonDebugger.h
PythonTracing.h
PythonEditor.h
SyntaxHighlighter.h
TextEdit.h

2
src/Gui/PreferencePages/DlgSettingsPythonConsole.cpp
View File

@@ -44,6 +44,7 @@ void DlgSettingsPythonConsole::saveSettings()
ui->PythonWordWrap->onSave();
ui->PythonBlockCursor->onSave();
ui->PythonSaveHistory->onSave();
ui->ProfilerInterval->onSave();
}
void DlgSettingsPythonConsole::loadSettings()
@@ -51,6 +52,7 @@ void DlgSettingsPythonConsole::loadSettings()
ui->PythonWordWrap->onRestore();
ui->PythonBlockCursor->onRestore();
ui->PythonSaveHistory->onRestore();
ui->ProfilerInterval->onRestore();
}
void DlgSettingsPythonConsole::changeEvent(QEvent *e)

34
src/Gui/PreferencePages/DlgSettingsPythonConsole.ui
View File

@@ -78,6 +78,35 @@ horizontal space in Python console</string>
</property>
</widget>
</item>
<item row="3" column="0">
<widget class="QLabel" name="labelProfilerInterval">
<property name="text">
<string>Python profiler interval (milliseconds): </string>
</property>
</widget>
</item>
<item row="3" column="1">
<widget class="Gui::PrefSpinBox" name="ProfilerInterval">
<property name="toolTip">
<string>The interval at which the profiler runs when there's Python code running (to keep the GUI responding). Set to 0 to disable.</string>
</property>
<property name="minimum">
<number>0</number>
</property>
<property name="maximum">
<number>5000</number>
</property>
<property name="value">
<number>200</number>
</property>
<property name="prefEntry" stdset="0">
<cstring>ProfilerInterval</cstring>
</property>
<property name="prefPath" stdset="0">
<cstring>PythonConsole</cstring>
</property>
</widget>
</item>
</layout>
</widget>
</item>
@@ -102,6 +131,11 @@ horizontal space in Python console</string>
<extends>QCheckBox</extends>
<header>Gui/PrefWidgets.h</header>
</customwidget>
<customwidget>
<class>Gui::PrefSpinBox</class>
<extends>QSpinBox</extends>
<header>Gui/PrefWidgets.h</header>
</customwidget>
</customwidgets>
<resources/>
<connections/>

50
src/Gui/PythonConsole.cpp
View File

@@ -32,6 +32,7 @@
# include <QTextCursor>
# include <QTextDocumentFragment>
# include <QTextStream>
# include <QTime>
# include <QUrl>
#endif
@@ -40,6 +41,7 @@
#include "PythonConsole.h"
#include "PythonConsolePy.h"
#include "PythonTracing.h"
#include "Application.h"
#include "CallTips.h"
#include "FileDialog.h"
@@ -116,12 +118,15 @@ struct PythonConsoleP
colormap[QLatin1String("Python error")] = Qt::red;
}
};
struct InteractiveInterpreterP
{
PyObject* interpreter;
PyObject* sysmodule;
PyObject* interpreter{nullptr};
PyObject* sysmodule{nullptr};
QStringList buffer;
PythonTracing trace;
};
} // namespace Gui
InteractiveInterpreter::InteractiveInterpreter()
@@ -295,6 +300,16 @@ bool InteractiveInterpreter::runSource(const char* source) const
return false;
}
bool InteractiveInterpreter::isOccupied() const
{
return d->trace.isActive();
}
bool InteractiveInterpreter::interrupt() const
{
return d->trace.interrupt();
}
/* Execute a code object.
*
* When an exception occurs, a traceback is displayed.
@@ -302,6 +317,13 @@ bool InteractiveInterpreter::runSource(const char* source) const
*/
void InteractiveInterpreter::runCode(PyCodeObject* code) const
{
if (isOccupied()) {
return;
}
d->trace.fetchFromSettings();
PythonTracingLocker tracelock(d->trace);
Base::PyGILStateLocker lock;
PyObject *module, *dict, *presult; /* "exec code in d, d" */
module = PyImport_AddModule("__main__"); /* get module, init python */
@@ -478,6 +500,10 @@ PythonConsole::PythonConsole(QWidget *parent)
d->output = d->info;
printPrompt(PythonConsole::Complete);
loadHistory();
flusher = new QTimer(this);
connect(flusher, &QTimer::timeout, this, &PythonConsole::flushOutput);
flusher->start(100);
}
/** Destroys the object and frees any allocated resources */
@@ -558,6 +584,12 @@ void PythonConsole::keyPressEvent(QKeyEvent * e)
QTextCursor cursor = this->textCursor();
QTextCursor inputLineBegin = this->inputBegin();
if (e->key() == Qt::Key_C && e->modifiers() == Qt::ControlModifier) {
if (d->interpreter->interrupt()) {
return;
}
}
if (!cursorBeyond( cursor, inputLineBegin ))
{
/**
@@ -733,6 +765,15 @@ void PythonConsole::onFlush()
printPrompt(PythonConsole::Flush);
}
void PythonConsole::flushOutput()
{
if (d->interpreter->isOccupied()) {
if (d->output.length() > 0 || d->error.length() > 0) {
printPrompt(PythonConsole::Complete);
}
}
}
/** Prints the ps1 prompt (>>> ) for complete and ps2 prompt (... ) for
* incomplete commands to the console window.
*/
@@ -823,6 +864,11 @@ void PythonConsole::runSource(const QString& line)
return;
}
if (d->interpreter->isOccupied()) {
insertPythonError(QString::fromLatin1("Previous command still running!"));
return;
}
bool incomplete = false;
Base::PyGILStateLocker lock;
PyObject* default_stdout = PySys_GetObject("stdout");

15
src/Gui/PythonConsole.h
View File

@@ -24,6 +24,7 @@
#define GUI_PYTHONCONSOLE_H
#include <Python.h>
#include <QTimer>
#include "PythonEditor.h"
@@ -42,6 +43,9 @@ public:
InteractiveInterpreter();
~InteractiveInterpreter();
bool isOccupied() const;
bool interrupt() const;
bool push(const char*);
int compileCommand(const char*) const;
bool hasPendingInput( ) const;
@@ -151,6 +155,7 @@ private:
void appendOutput(const QString&, int);
void loadHistory() const;
void saveHistory() const;
void flushOutput();
Q_SIGNALS:
void pendingSource( );
@@ -158,13 +163,13 @@ Q_SIGNALS:
private:
struct PythonConsoleP* d;
PythonConsoleHighlighter* pythonSyntax{nullptr};
QString *_sourceDrain{nullptr};
QString _historyFile;
QTimer *flusher{nullptr};
friend class PythonStdout;
friend class PythonStderr;
private:
PythonConsoleHighlighter* pythonSyntax;
QString *_sourceDrain;
QString _historyFile;
};
/**

161
src/Gui/PythonTracing.cpp Normal file
View File

@@ -0,0 +1,161 @@
// SPDX-License-Identifier: LGPL-2.1-or-later
/***************************************************************************
* Copyright (c) 2023 Werner Mayer <wmayer[at]users.sourceforge.net> *
* *
* This file is part of FreeCAD. *
* *
* FreeCAD is free software: you can redistribute it and/or modify it *
* under the terms of the GNU Lesser General Public License as *
* published by the Free Software Foundation, either version 2.1 of the *
* License, or (at your option) any later version. *
* *
* FreeCAD is distributed in the hope that it will be useful, but *
* WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
* Lesser General Public License for more details. *
* *
* You should have received a copy of the GNU Lesser General Public *
* License along with FreeCAD. If not, see *
* <https://www.gnu.org/licenses/>. *
* *
**************************************************************************/
#include "PreCompiled.h"
#ifndef _PreComp_
#include <QTime>
#include <QGuiApplication>
#endif
#include "PythonTracing.h"
#include <App/Application.h>
using namespace Gui;
struct PythonTracing::Private
{
bool active{false};
int timeout{200}; //NOLINT
// NOLINTBEGIN
static int profilerInterval;
static bool profilerDisabled;
// NOLINTEND
};
// NOLINTBEGIN
int PythonTracing::Private::profilerInterval = 200;
bool PythonTracing::Private::profilerDisabled = false;
// NOLINTEND
PythonTracing::PythonTracing()
: d{std::make_unique<Private>()}
{
}
PythonTracing::~PythonTracing() = default;
bool PythonTracing::isActive() const
{
return d->active;
}
void PythonTracing::activate()
{
d->active = true;
setPythonTraceEnabled(true);
}
void PythonTracing::deactivate()
{
d->active = false;
setPythonTraceEnabled(false);
}
void PythonTracing::fetchFromSettings()
{
const long defaultTimeout = 200;
auto parameterGroup = App::GetApplication().GetParameterGroupByPath(
"User parameter:BaseApp/Preferences/PythonConsole");
int interval = static_cast<int>(parameterGroup->GetInt("ProfilerInterval", defaultTimeout));
setTimeout(interval);
}
bool PythonTracing::interrupt() const
{
if (isActive()) {
PyErr_SetInterrupt();
return true;
}
return false;
}
void PythonTracing::setTimeout(int ms)
{
d->timeout = ms;
}
int PythonTracing::timeout() const
{
return d->timeout;
}
void PythonTracing::setPythonTraceEnabled(bool enabled) const
{
Py_tracefunc trace = nullptr;
if (enabled && timeout() > 0) {
Private::profilerInterval = timeout();
trace = &tracer_callback;
}
else {
Private::profilerDisabled = true;
}
PyEval_SetTrace(trace, nullptr);
}
/*
* This callback ensures that Qt runs its event loop (i.e. updates the GUI, processes keyboard and
* mouse events, etc.) at least every 200 ms, even when there is long-running Python code on the
* main thread. It is registered as the global trace function of the Python environment.
*/
int PythonTracing::tracer_callback(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg)
{
Q_UNUSED(obj)
Q_UNUSED(frame)
Q_UNUSED(what)
Q_UNUSED(arg)
static QTime lastCalledTime = QTime::currentTime();
QTime currTime = QTime::currentTime();
// if previous code object was executed
if (Private::profilerDisabled) {
Private::profilerDisabled = false;
lastCalledTime = currTime;
}
int ms = lastCalledTime.msecsTo(currTime);
if (ms >= Private::profilerInterval) {
lastCalledTime = currTime;
QGuiApplication::processEvents();
}
return 0;
}
// ------------------------------------------------------------------------------------------------
PythonTracingLocker::PythonTracingLocker(PythonTracing& trace)
: trace{trace}
{
trace.activate();
}
PythonTracingLocker::~PythonTracingLocker()
{
trace.deactivate();
}

118
src/Gui/PythonTracing.h Normal file
View File

@@ -0,0 +1,118 @@
// SPDX-License-Identifier: LGPL-2.1-or-later
/***************************************************************************
* Copyright (c) 2023 Werner Mayer <wmayer[at]users.sourceforge.net> *
* *
* This file is part of FreeCAD. *
* *
* FreeCAD is free software: you can redistribute it and/or modify it *
* under the terms of the GNU Lesser General Public License as *
* published by the Free Software Foundation, either version 2.1 of the *
* License, or (at your option) any later version. *
* *
* FreeCAD is distributed in the hope that it will be useful, but *
* WITHOUT ANY WARRANTY; without even the implied warranty of *
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU *
* Lesser General Public License for more details. *
* *
* You should have received a copy of the GNU Lesser General Public *
* License along with FreeCAD. If not, see *
* <https://www.gnu.org/licenses/>. *
* *
**************************************************************************/
#ifndef GUI_PYTHONTRACING_H
#define GUI_PYTHONTRACING_H
#include <Python.h>
#include <frameobject.h>
#include <memory>
#include <FCGlobal.h>
namespace Gui {
class GuiExport PythonTracing
{
public:
PythonTracing();
~PythonTracing();
PythonTracing(const PythonTracing&) = delete;
PythonTracing(PythonTracing&&) = delete;
PythonTracing& operator = (const PythonTracing&) = delete;
PythonTracing& operator = (PythonTracing&&) = delete;
/*!
* \brief isActive
* Returns true if the tracing is active, false otherwise.
* \return
*/
bool isActive() const;
/*!
* \brief activate
* Activates the Python tracing.
*/
void activate();
/*!
* \brief deactivate
* Deactivates the Python tracing.
*/
void deactivate();
/*!
* \brief fetchFromSettings
* Fetch parameters from user settings.
*/
void fetchFromSettings();
/*!
* \brief interrupt
* If the tracing is enabled it interrupts the Python interpreter.
* True is returned if the tracing is active, false otherwise.
*/
bool interrupt() const;
/*!
* \brief setTimeout
* Sets the interval after which the Qt event loop will be processed.
*/
void setTimeout(int ms);
/*!
* \brief timeout
* \return the timeout of processing the event loop.
*/
int timeout() const;
private:
void setPythonTraceEnabled(bool enabled) const;
static int tracer_callback(PyObject *obj, PyFrameObject *frame, int what, PyObject *arg);
private:
struct Private;
std::unique_ptr<Private> d;
};
class GuiExport PythonTracingLocker
{
public:
/*!
* \brief PythonTracingLocker
* Activates the passed tracing object.
* \param trace
*/
explicit PythonTracingLocker(PythonTracing& trace);
/*!
* Deactivates the tracing object.
*/
~PythonTracingLocker();
PythonTracingLocker(const PythonTracingLocker&) = delete;
PythonTracingLocker(PythonTracingLocker&&) = delete;
PythonTracingLocker& operator = (const PythonTracingLocker&) = delete;
PythonTracingLocker& operator = (PythonTracingLocker&&) = delete;
private:
PythonTracing& trace;
};
} // namespace Gui
#endif // GUI_PYTHONTRACING_H