Tobias Diekershoff
1bf089e424
Merge pull request #3216 from annando/1701-performance
...
Better network handling, improved structure
2017-03-12 15:40:56 +01:00
Michael
0afb0c2ea4
NULL_DATE is now a constant
2017-02-27 23:37:15 +00:00
Michael
99fb15037b
Removed some more unneeded stuff
2017-02-27 06:27:22 +00:00
Michael
48209f0ecd
Now there is only the worker.
2017-02-26 23:16:49 +00:00
Michael
da8a48f89e
Bugfix: the curl option CURLOPT_SSL_VERIFYHOST is problematic
2017-02-25 13:59:24 +00:00
Michael
2424cb78d1
Only process feed and dfrn items when they aren't already stored
2017-01-31 19:39:09 +00:00
Michael
64d06b98b2
Added documentation
2017-01-27 15:13:37 +00:00
Michael
69f1deb166
Some added logging
2017-01-21 20:15:49 +00:00
Michael
dc439c6e50
Bugfix for masses of php warnings
2017-01-21 19:50:56 +00:00
Michael
c74b7565a9
Rearranged the logging
2017-01-21 06:16:15 +00:00
Michael
45d6f6c0a3
Some changed logging
2017-01-21 06:06:29 +00:00
Michael
72884d75d5
Added callstack
2017-01-21 05:05:41 +00:00
Michael
8939a25502
Better number format
2017-01-20 22:05:50 +00:00
Michael
9d77e91f5f
Some more logging
2017-01-20 21:58:53 +00:00
Michael
1360a0a003
The worker now tells the process runtime length.
2017-01-19 17:06:23 +00:00
Roland Haeder
7c342600c3
used more App::get_baseurl() instead of get_app()->get_baseurl().
...
Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-12-20 21:51:25 +01:00
Roland Häder
a96eb3428d
Used more dbm::is_result() instead of (!$r) or (!count($r)), still there are
...
more pending ...
Also put SQL table columns into back-ticks.
Signed-off-by: Roland Häder <roland@mxchange.org>
2016-12-19 14:46:33 +01:00
Roland Häder
293436e5fd
Some rewrites:
...
- max_proccesses_reach() and maxload_reached() (why no _ behind max?) are called
both way, static and with object reference.
- this is strongly discouraged and should be avoided as its support (in PHP) may
be dropped in future releases.
- used $a = get_app(); to encapsulate code (even when the function does
currently the same, it may be changed later)
Signed-off-by: Roland Häder <roland@mxchange.org>
2016-12-13 10:16:36 +01:00
Michael
6845775ff9
Store spooled data
2016-12-01 20:53:18 +00:00
Michael
57c95c93e1
The frontend worker is now working as a full replacement for the crontab.
2016-11-30 19:24:58 +00:00
Michael
cc5eebd173
The frontend worker can now fork background processes
2016-11-29 22:40:19 +00:00
Michael
636a033364
Added documentation, config stuff, ...
2016-11-27 09:02:08 +00:00
Michael
d71321c834
Some added documentation
2016-11-27 01:11:22 +00:00
Michael
e856ab9a09
The worker can now run from the frontend as well
2016-11-27 00:55:05 +00:00
Michael Vogel
47b8975bb6
poller.php is now working with transactions as well.
2016-10-23 23:31:56 +00:00
Michael Vogel
3e5cf5290e
Improved queries, more uncommitted queries
2016-10-17 18:38:51 +00:00
Michael Vogel
a47ae788ba
Improved logging
2016-10-11 06:30:53 +00:00
Michael Vogel
ef6a23caf3
Worker: Reserve some space for low priority processes
2016-10-10 21:01:36 +00:00
Michael Vogel
6abac720e2
Small performance tweaks for "item_store" as well.
2016-10-07 06:05:43 +00:00
Michael Vogel
6d16c39b7e
dbstructure now switches in the maintenance mode when updating
2016-10-02 13:52:52 +00:00
Michael Vogel
1caa87dce6
Restructured the way, processes are inserted and removed
2016-09-09 20:55:49 +00:00
Michael Vogel
22f32d9721
New process table for a better detection of running workers
2016-09-09 20:33:54 +00:00
Michael Vogel
108a1c4ac6
New priority level for stale processes.
2016-08-18 12:33:17 +02:00
Michael Vogel
67b16507f1
Performance improvements for the poller
2016-08-14 21:02:29 +02:00
Michael Vogel
d138b11868
Use a 10 minute timeout for high priority processes. This may be better.
2016-08-08 21:28:44 +02:00
Michael Vogel
29168de677
Process timeouts are now priority depending
2016-08-08 19:20:40 +02:00
Michael Vogel
0411eb289e
Lower the priority for stale processes
2016-08-04 15:41:32 +02:00
Michael Vogel
074ae59f49
Reschedule killed processes at the beginning of the queue.
2016-08-04 15:33:15 +02:00
Michael Vogel
7157ca3809
Fastlane is now working with any priority level
2016-08-03 15:59:25 +02:00
Michael Vogel
ff430640c0
Small variable type fixed, removed unused includes
2016-08-03 10:19:46 +02:00
Michael Vogel
d673f44c5b
Split cron jobs in cronjobs, introduce fastlane for high priority tasks
2016-08-03 10:03:05 +02:00
Michael Vogel
b9dbb0ace1
Split cronhook call to several single calls
2016-08-02 06:28:34 +02:00
Michael Vogel
668da905e2
"proc_run" is now called with priority.
2016-08-01 07:48:43 +02:00
Michael Vogel
5e81c105e0
We now work with the "priority" field - that's better
2016-07-24 20:25:11 +02:00
Michael Vogel
4be7e5a581
"Fast lane" mechanism to reserve one queue for priority processes
2016-07-24 11:22:44 +02:00
Michael Vogel
7580c8ed11
Processing of high priority processes
2016-07-23 22:57:22 +02:00
Michael Vogel
fb447e84ee
Some more information about the current queue status of the worker
2016-07-22 18:07:04 +02:00
Michael Vogel
4b29937a74
Worker: New value for cooldown time after each function call
2016-07-13 19:43:16 +02:00
Tobias Diekershoff
598a9f76a6
Merge pull request #2658 from annando/1607-maxload
...
Central Backend/Frontend detection when checking for overload
2016-07-02 21:52:48 +02:00
Michael Vogel
e4c6cfd2d8
Moves the function "is_result" from dba to dbm.
2016-07-02 14:00:42 +02:00
Michael Vogel
7d531afd00
Merge branch 'rewrites/is_filled_array_introduced' of git://github.com/Quix0r/friendica into Quix0r-rewrites/is_filled_array_introduced
...
Conflicts:
boot.php
2016-07-02 13:38:23 +02:00
Michael Vogel
c549ae939c
Added the new function to the poller.
2016-07-02 09:31:28 +02:00
Michael Vogel
0518f004c2
removed duplicated functions
2016-06-04 12:20:09 +02:00
Michael Vogel
b45772ba6e
Configurable amount of maximum processes
2016-06-04 11:04:26 +02:00
Michael Vogel
df337e57d1
Quit when processlist is too long
2016-06-03 23:10:23 +02:00
Michael Vogel
85a9c7d96e
The poller now has the logging for processes as well.
2016-06-01 21:54:02 +02:00
Michael Vogel
76b24f61a9
New (experimental) value to define the maximum level of database connections for the worker
2016-04-23 10:11:09 +02:00
Roland Haeder
9cfc249b12
Moved is_filled_array() to both dba classes and named it is_result().
...
Please see ticket #2390 for full discussion.
Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-03-12 18:54:55 +01:00
Roland Haeder
c60605ce0c
Merge branch 'develop' of github.com:friendica/friendica into rhaeder-develop
2016-03-12 18:50:14 +01:00
Michael Vogel
beb2346cfc
The function to check for maxload and the lockfile is centralized
2016-03-08 20:28:09 +01:00
Roland Haeder
d88606321d
Just easier code ...
...
Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-03-04 22:39:08 +01:00
Roland Haeder
d5c1ce490b
No processing if error or empty array
...
Signed-off-by: Roland Haeder <roland@mxchange.org>
2016-03-04 22:38:18 +01:00
Michael Vogel
acb09d3a3d
Database connections: When we now check for user values we check the system values as well
2016-02-12 11:04:25 +01:00
Michael Vogel
5027fbf733
Poller: Now the user limits can be detected automatically as well
2016-02-11 21:39:34 +01:00
Michael Vogel
4af77b60ce
Poller: Use the processlist when the number of maximum database connections was provided manually
2016-02-11 11:33:45 +01:00
Michael Vogel
211be5bfc5
Poller: The maximum number of connections can now be configured
2016-02-09 23:28:33 +01:00
Michael Vogel
7af3dd01d8
Poller: Check the number of used database connections
2016-02-09 06:42:00 +01:00
Michael Vogel
275f7b2660
The worker now kills processes that run longer than 3 hours
2016-01-24 02:53:21 +01:00
rabuzarus
3df5c17124
reformat todo according to doxygen style
2015-12-25 23:17:34 +01:00
Michael Vogel
33f354a68c
Bugfix for the maximum load check in worker.
2015-12-16 00:14:53 +01:00
Michael Vogel
6e03477598
Sometimes the function "sys_getloadavg" doesn't return an array. This is a workaround.
2015-12-15 23:26:58 +01:00
Michael Vogel
fbbba6828b
Some better logging
2015-12-06 20:04:33 +01:00
Michael Vogel
9f8da37c99
Move the process check at the beginning of the script
2015-12-06 16:40:31 +01:00
Michael Vogel
54085508e5
Double check for maximum number of workers
2015-12-06 16:28:28 +01:00
Michael Vogel
c3ee255b0c
Reduction of double executions - hopefully
2015-09-29 06:15:26 +02:00
Michael Vogel
ed8cdc7d85
Logging message added
2015-09-28 22:32:56 +02:00
Michael Vogel
f0cf9ce519
Prevent double execution
2015-09-28 21:58:58 +02:00
Michael Vogel
eb75d9532b
Magical mathematics to reduce the number of workers
2015-09-28 19:14:07 +02:00
Michael Vogel
9767801be4
Load depending number of worker queues.
2015-09-28 07:54:28 +02:00
Michael Vogel
ae21c40f21
Load depending number of workers
2015-09-27 13:56:20 +02:00
Michael Vogel
173d1390df
Mute warnings in pidfile/Quit poller after an hour.
2015-09-25 17:38:56 +02:00
Michael Vogel
f8e4a71eda
Do a load check during execution of the queue.
2015-09-23 08:56:48 +02:00
Michael Vogel
3ace2136f0
Checking includes for valid paths
2015-09-13 18:47:10 +02:00
Michael Vogel
12659fc3a1
Let the cronhook be called different from the cron job.
2015-09-12 20:22:58 +02:00
Michael Vogel
7edce8e266
Don't use a delivery interval when using the worker
2015-09-12 18:08:03 +02:00
Michael Vogel
0a2c161230
Fork as many processes as possible from the start on.
2015-09-12 17:51:27 +02:00
Michael Vogel
32e8f3468d
Moved some functionality back to the cron. Speed up things
2015-09-11 21:35:58 +02:00
Michael Vogel
ff739b0a23
Just changed some script names
2015-09-10 23:32:56 +02:00
Michael Vogel
d3a6ebfe7e
The worker is now working
2015-09-10 23:10:31 +02:00
Michael Vogel
b0ddde22c0
By entering a GNU Social account name, the friends of this contact will be added to the friendica account
2015-08-23 22:21:15 +02:00
Michael Vogel
820ded2c2d
Support for the new nodeinfo protocol that will replace statistics.json
2015-08-10 21:33:57 +02:00
Michael Vogel
166519fc4d
Move the contact discovery into the background
2015-07-26 14:41:34 +02:00
Michael Vogel
cf3214c904
Query other servers periodically for their known contacts
2015-07-18 22:26:06 +02:00
Michael Vogel
bffe35ab71
There is now two different checks for the completion of conversations
2015-06-29 21:53:59 +02:00
Michael Vogel
ae172a9cec
If the load is too high, now not only the poller but also the page delivery will stop.
2015-03-15 07:45:50 +01:00
Michael Vogel
dfdb804323
Beautified the poller logging
2015-02-19 22:38:02 +01:00
Michael Vogel
ab5bf06873
Restructured and simplified the poller check
2015-02-19 22:30:16 +01:00
Michael Vogel
68c9707221
The poller now always respects the priority setting for feeds.
2015-02-19 22:04:05 +01:00
Michael Vogel
4c901e60cc
More logging for the poller. Changed the query for the contacts to reduce the amount of contacts
2015-02-19 21:36:29 +01:00
Michael Vogel
6d8c126179
Do the directory change only when it is needed.
2015-02-19 10:45:46 +01:00
