Commit graph

49 commits

Author SHA1 Message Date
Kumi d24a94dc7f Fix copyright blocks 2018-01-03 12:17:50 +01:00
Riccardo Spagni 87d332df1a
Merge pull request #2385
2289ead5 blockchain_import: warn for chunks over 500000, not 100000 (moneromooo-monero)
2f9a5528 blockchain_import: properly cleanup core/db on exit (moneromooo-monero)
8d6967be blockchain_import: do not error out on truncated files (moneromooo-monero)
2017-09-02 11:35:12 +02:00
moneromooo-monero 2289ead568
blockchain_import: warn for chunks over 500000, not 100000
We have a lot of 350000 byte blocks now.
2017-09-01 22:50:15 +01:00
moneromooo-monero 2f9a5528a7
blockchain_import: properly cleanup core/db on exit 2017-09-01 12:01:36 +01:00
moneromooo-monero 8d6967be39
blockchain_import: do not error out on truncated files
This will happen often when downloading from a live file
2017-09-01 12:01:33 +01:00
moneromooo-monero cf4aa65316
Fix blockchain_import wedge on exception in cleanup_handle_incoming_blocks 2017-08-29 15:45:08 +01:00
Howard Chu 80344740bd
More DB support cleanup
Hide DB types from db_types.h - no reason to recompile dependencies
when DB types change.

Also remove lingering in-memory DB references, they've been
obsolete since 9e82b694da
2017-08-20 13:57:36 +01:00
Howard Chu 4c7f8ac04f
DB cleanup
Hide LMDB-specific stuff behind blockchain_db.h. Nobody besides blockchain_db.cpp
should ever be including DB-specific headers any more.
2017-08-19 18:11:38 +01:00
moneromooo-monero c93b7692b4
blockchain_import: much faster when verifying with cryptonote::core
Quick test with the first 56569 blocks from mainnet

version verify  batch   time
old     0       200     1:16
new     0       200     0:57
old     0       5000    0:53
new     0       5000    0:51
old     1       200     est > 1h
new     1       200     10:21
old     1       5000    est > 1h
new     1       5000    8:27
2017-08-07 14:32:49 +01:00
moneromooo-monero b52abd1370
Move txpool to the database
Integration could go further (ie, return_tx_to_pool calls should
not be needed anymore, possibly other things).

poolstate.bin is now obsolete.
2017-05-25 22:23:37 +01:00
moneromooo-monero 3835903650
blockchain_import: make --log-level understand categories 2017-03-05 19:45:22 +00:00
moneromooo-monero 50ebf66150
blockchain_import: error out nicely on exceptions 2017-03-05 19:25:01 +00:00
Howard Chu e4437b2551
Fix double-import
Incomplete ifdef cleanup in 9e82b694da
2017-03-03 21:45:14 +00:00
Riccardo Spagni c3599fa7b9
update copyright year, fix occasional lack of newline at line end 2017-02-21 19:38:18 +02:00
moneromooo-monero 882db8e9d9
tools: log to the correct file 2017-02-12 19:39:59 +00:00
moneromooo-monero 142e5e1eb4
tools: set requested log level
also move a couple logs from INFO to DEBUG
2017-02-12 18:48:39 +00:00
kenshi84 8027ce0c75 extract some basic code from libcryptonote_core into libcryptonote_basic 2017-02-08 22:45:15 +09:00
moneromooo-monero a480bf6b14
fixups in logging init calls, and add missing net context in a log 2017-01-16 10:45:59 +00:00
moneromooo-monero 5833d66f65
Change logging to easylogging++
This replaces the epee and data_loggers logging systems with
a single one, and also adds filename:line and explicit severity
levels. Categories may be defined, and logging severity set
by category (or set of categories). epee style 0-4 log level
maps to a sensible severity configuration. Log files now also
rotate when reaching 100 MB.

To select which logs to output, use the MONERO_LOGS environment
variable, with a comma separated list of categories (globs are
supported), with their requested severity level after a colon.
If a log matches more than one such setting, the last one in
the configuration string applies. A few examples:

This one is (mostly) silent, only outputting fatal errors:

MONERO_LOGS=*:FATAL

This one is very verbose:

MONERO_LOGS=*:TRACE

This one is totally silent (logwise):

MONERO_LOGS=""

This one outputs all errors and warnings, except for the
"verify" category, which prints just fatal errors (the verify
category is used for logs about incoming transactions and
blocks, and it is expected that some/many will fail to verify,
hence we don't want the spam):

MONERO_LOGS=*:WARNING,verify:FATAL

Log levels are, in decreasing order of priority:
FATAL, ERROR, WARNING, INFO, DEBUG, TRACE

Subcategories may be added using prefixes and globs. This
example will output net.p2p logs at the TRACE level, but all
other net* logs only at INFO:

MONERO_LOGS=*:ERROR,net*:INFO,net.p2p:TRACE

Logs which are intended for the user (which Monero was using
a lot through epee, but really isn't a nice way to go things)
should use the "global" category. There are a few helper macros
for using this category, eg: MGINFO("this shows up by default")
or MGINFO_RED("this is red"), to try to keep a similar look
and feel for now.

Existing epee log macros still exist, and map to the new log
levels, but since they're used as a "user facing" UI element
as much as a logging system, they often don't map well to log
severities (ie, a log level 0 log may be an error, or may be
something we want the user to see, such as an important info).
In those cases, I tried to use the new macros. In other cases,
I left the existing macros in. When modifying logs, it is
probably best to switch to the new macros with explicit levels.

The --log-level options and set_log commands now also accept
category settings, in addition to the epee style log levels.
2017-01-16 00:25:46 +00:00
moneromooo-monero 6da9335a9e
blockchain_import: fix build after tx_pool::add_tx changes 2017-01-15 21:52:56 +00:00
moneromooo-monero 9e82b694da
remove original Cryptonote blockchain_storage blockchain format 2016-08-28 21:27:32 +01:00
moneromooo-monero f7301c3563
Revert "Print stack trace upon exceptions"
Ain't nobody got time for link/cmake skullduggery.

This reverts commit fff238ec94.
2016-03-21 10:12:23 +00:00
moneromooo-monero fff238ec94
Print stack trace upon exceptions
Useful for debugging users' logs
2016-03-19 21:48:36 +00:00
moneromooo-monero 5e3557d2c3
move g_test_dbg_lock_sleep from a global to a function level static
This avoids the need to define that variable in every program
which uses epee.
2016-02-22 19:34:09 +00:00
warptangent 05e7ac0360
blockchain_import: Check bit width for more than just WIN32
Pass the CMake bit width setting to compile flags for blockchain_import
and blockchain_converter.

For LMDB on 32-bit, hyc has found that batch size of 100 appears to be a
good default.
2016-02-17 12:37:15 -08:00
warptangent 27f76e2ee4
blockchain_import: Build string for db type list 2016-02-14 16:25:35 -08:00
warptangent 1aa8a9d26f
blockchain_import: Add mode argument representing multiple DB flags
These modes match those optionally provided as part of the daemon's
--db-type argument.

Argument after the # is interpreted as a composite mode if there's only
one (no comma separated arguments).

Sample usage:

blockchain_import --database lmdb#fastest

blockchain_import --database berkeley#fastest

Multiple specific DB flags are still supported, e.g.

blockchain_import --database lmdb#nosync,nordahead

blockchain_import --database berkeley#txn_nosync
2016-02-14 16:25:35 -08:00
warptangent cffc411c90
blockchain_import: Support BerkeleyDB 2016-02-14 16:02:38 -08:00
warptangent 19c1aaa9f6
blockchain_import: Add database type argument
Default to LMDB.
2016-02-14 16:02:37 -08:00
warptangent 0fedce00d5
blockchain_import: Make LMDB-specific names general
Also update terms to better distinguish blockchain management/engine
from database type.
2016-02-14 16:02:37 -08:00
warptangent b368e29f67
blockchain_import: Get hard fork version from HardFork
Replace temporary assignment that used hardcoded values.
2016-02-08 09:28:21 -08:00
warptangent e02577f594
Move HardFork DB update to BlockchainDB::add_block()
Ensures the database is consistent.

Also simplifes blockchain_import in that verify mode off has less to
work around.
2016-02-08 09:28:19 -08:00
warptangent 9d62c161a3
blockchain_import: Add hard fork data for each block when verify mode is off 2016-02-08 08:50:59 -08:00
warptangent c657e772c4
blockchain_import: Add --drop-hard-fork command 2016-02-08 08:50:47 -08:00
warptangent 412243901d
blockchain_import: Add exception to log output 2016-02-08 08:50:27 -08:00
moneromooo-monero 3b1d7e03fc
Fix V1/V2 use of hard fork related parameters
Some of it uses hardcoded height, which will need some thinking
for next (voted upon) fork.
2016-01-29 17:21:11 +00:00
Riccardo Spagni 5c71f5d13e
update version 2015-12-31 23:12:13 +02:00
Riccardo Spagni de03926850
updated copyright year 2015-12-31 08:39:56 +02:00
moneromooo-monero 3f2970fadb
Add missing semicolons after log statements 2015-12-19 14:47:37 +00:00
Riccardo Spagni 9276233b9e
Merge pull request #493
f3724ae Fix startup crash when using a locale boost does not like (moneromooo-monero)
0c1dae3 i18n: allow language to be passed as a parameter (moneromooo-monero)
2015-11-21 13:08:42 +02:00
moneromooo-monero f3724aef88
Fix startup crash when using a locale boost does not like
There are various locale related bugs in various versions of boost,
where exceptions are thrown in boost::filesystem APIs when the
current locale is not to boost's liking. It's not clear what "not
to boost's liking" means in detail, though "en" and "en_US.UTF-8"
are not to its liking.

Fix it by running a test function that's known to throw in such
a case, and resetting LANG and LC_ALL to C if an exception is
thrown. In simplewallet, the locale is queried before that so the
correct translations will still be used.
2015-11-21 10:13:10 +00:00
moneromooo-monero 932994c0cb
Relay transactions when they linger too long in the pool
The last relayed time of a transaction is maintained, and
transactions will be relayed again if they are still in the
pool after a certain amount of time, which increases with
the transaction's age. All such transactions are resent,
whether or not they originated on the local node.
2015-11-21 00:56:21 +00:00
warptangent 471e8a3195
blockchain_import: Add --input-file option
This option specifies the input file path for importing.

The default remains <data-dir>/export/blockchain.raw
2015-08-14 15:00:34 -07:00
warptangent 3ffda6eefb
blockchain_import: Updates for naming consistency 2015-08-14 15:00:18 -07:00
warptangent 699e4b3f65
blockchain_utilities: Pass expected number of blocks when starting batch 2015-07-11 23:54:14 -07:00
warptangent d1eac1b71c
Support debugging command --pop-blocks on in-memory blockchain
Add public method blockchain_storage::debug_pop_block_from_blockchain()

Ensure blockchain_import calls destructors before exit.

To test:

DATABASE=memory make release

// create blockchain.bin from blockchain.raw if needed
build/release/bin/blockchain_import --block-stop 1000

// try popping a single block
build/release/bin/blockchain_import --pop-blocks 1
2015-05-16 19:38:52 -07:00
warptangent 73d3511412
Rename "--block-number" option to "--block-stop"
Update help output for this and other options.
2015-05-15 17:26:43 -07:00
warptangent 2b2dbd887d
Check if chunk size is zero instead of negative
This corrects an unnecessary check and fixes compile error on OS X.
2015-05-15 17:23:01 -07:00
warptangent ac011b4312
Rename src/blockchain_converter/ to src/blockchain_utilities/
Update appropriate files (CMakeLists.txt, README.md)
2015-05-08 14:24:51 -07:00
Renamed from src/blockchain_converter/blockchain_import.cpp (Browse further)